Wednesday, December 24, 2008

Evaluating Simon - Java monitoring

Recently a new monitoring kid appeared on the block: Java Simon (not to be confused with Dejal's Simon or some other simons).

Simon claims to be the successor of JAMon. As I just started a project to improve the documentation of Jamon, and integrate this better with Wicket projects, I thought this would be a good time to evaluate Simon and also to compare it to Jamon. Here are the results.

Features
Simon (only just started) can measure task durations (min, max, count, maximum concurrency and some timestamps) and has task counters. Other statistics can be added (supported now: standard deviation). The JDBC proxy driver can be used to monitor database usage. The monitors are grouped in a tree. You can disable each monitor individually, or per complete subtree.

Jamon adds the ability to measure in any unit (for example euro value of a transaction) and has quite a few monitors for Tomcat, Jetty, etc. Jamon does not organize monitors in a tree, so monitors can be disabled individually, or all at the same time.

So Jamon clearly wins on the feature front, but if you're just into the basic stuff Simon is fine.

Documentation
One of the weak points of Jamon is its documentation (hence my little project). Simon on the other hand starts with a good set of wiki pages. If these are maintained properly, Simon is the clear winner here.

Simon interface
Simon's interface is fairly simple. One gets a monitor through the singleton SimonManager, and call start and stop on it:

Stopwatch simon = SimonManager.getStopwatch("simon.test").start(); // Do something simon.stop();
(I will not discuss the Jamon interface further, but making a measurement is virtually the same.)

Getting data out of a Simon monitor (a simon) is a bit more difficult. You can just get the measurements like this:

Stopwatch simon = SimonManager.getStopwatch("simon.test"); long hitCount = simon.getCounter(); long max = simon.getMax(); long total = simon.getTotal(); simon.reset();
But that would be very thread unsafe. As each method is synchronized separately, the three measurements may represent different moments in time.

The idiomatic solution would be to use method sample:

Stopwatch simon = SimonManager.getStopwatch("simon.test"); Map<String, String> sample = simon.sample(true); long hitCount = Long.parseLong(sample.get("counter")); long max = Long.parseLong(sample.get("max")); long total = Long.parseLong(sample.get("total"));
This is more how it should be except for the boolean flag [F3, N7], and the return type of sample: a map! First of all, maps are very memory hungry [G21], secondly I have to assume that the values are longs [CE6] and parse them back from a String [GE37], and thirdly the keys are magic values [CE6, G25].

Here is another approach:

Stopwatch simon = SimonManager.getStopwatch("simon.test"); long hitCount; long max; long total; synchronized (simon) { hitCount = simon.getCounter(); max = simon.getMax(); total = simon.getTotal(); simon.reset(); }
Ouch. This works but uses client-side locking. That this is possible at all is a mistake in the Java language (as admitted by James Gosling, sorry, I lost the reference). Even so, Simon should not make it possible for clients to get the same lock that is used to protect the monitor data. This should be the only way to get actual data.

Recommendation: move all data retrieval methods (getMax(), getTotal(), etc.) from the monitor to an immutable data class, e.g. StopwatchMeasurements. Method sample should return instances of this class, or subclasses when a different analyzer was configured.

Update 2008-12-25: added following section on visiting all simons.
Update 2009-01-0: the iterator now actually works.

As I am using Jarep to browse through historic measurements, I want to expose the Simon data just like Jarep's servlet exposer for Jamon does. In other words: we need to visit each simon and extract the data as described above.

There are 2 approaches to visit all nodes. First is to get all simon names from SimonManager and then request them one by one. The other is to walk the tree of simons. I choose the latter. Unfortunately there is no tree iterator so I created one with commons-collections:

Simon rootSimon = SimonManager.getRootSimon(); ObjectGraphIterator simonIterator = new ObjectGraphIterator(rootSimon, new Transformer() { private Object previous; public Object transform(Object input) { if (input == previous) { previous = null; return input; } else { previous = input; Simon currentSimon = (Simon) input; return new IteratorChain( new SingletonIterator(currentSimon), currentSimon.getChildren().iterator()); } }});
Admittedly ugly, but it works. Still there is one glaring bug here. The children are stored in a regular TreeSet. When we request the children through getChildren we get a collection that wraps the original set. Unfortunately that means that when an simon is added while we iterate the complete simon tree, we get a ConcurrentModificationException! Not quite acceptable.

Recommendation: store children of a simon in a CopyOnWriteArrayList instead of a TreeSet. Another option is change the complete set of simons in the manager into a ConcurrentMap and provide an iterator over that.

Performance To measure the performance of Simon and Jamon I wrote a test application that mimics a random web applications. It has a variable number of threads to mimic the number of concurrent requests that are being handled. There is one monitor to measure the duration of the entire request, one monitor to measure a specific request, one monitor to measure the service layer, and 0 to 3 monitors to measure the database layer.

The test application was run from IntelliJ with Soy Latte 1.0.3 (a 1.6 JDK, part of the OpenJDK for Debian) on a 2.4GHz Intel Core 2 Duo.

The measurements below display the time spend in obtaining, starting and stopping a single monitor. Measurements were done in nanoseconds, but are displayed here in milliseconds (logarithmic scale).

Simon's performance is pretty good! The overhead in my test application doesn't come anywhere near the millisecond range even in highly concurrent situations. Jamon is not so bad either, even though its 5 to 200 times slower, depending on thread contention. With 5 monitors in a request, the overhead per request slowly grows to 0.7 ms for Simon, and quickly grows to 12ms for Jamon.

Note: the test application does nothing else but getting monitors and doing measurements. So the right side of the chart is unrealistic for most environments.

Improving the performance
Even though the performance is pretty good. I think it can be improved further. I investigated two potential performance improvements:

  • Remove the ThreadLocal variable in StopWatchImpl.
  • Replace synchronized HashMap with a ConcurrentHashMap in EnabledManager.
Remove the ThreadLocal variable in StopWatchImpl Stopwatch method start stores the start time in a thread local. When you call stop the start time is retrieved and used to calculate the duration.

ThreadLocals have seen an enormous performance improvement in recent JVMs, but you are still better off without.

The idea is as follows: method start will no longer return the monitor itself but an object that contains only the start time and a stop method. Lets call it a StopwatchMeasurement. Only in the stop method will it be necessary to synchronize and update the counters.
The advantages are:

  • We no longer need to synchronize the start method.
  • We no longer need the thread local variable to store the start time.
  • We no longer need the start(key) and stop(key) variants.
  • The stopwatch is no longer susceptible for memory leak when a malicious client keeps starting monitors in new threads, but never stops them (someone forgot a finally).
  • The stopwatch is no longer susceptible for memory leak when a malicious client keeps starting monitors with start(key) with a new key, but never stops them (again, someone forgot a finally).
Disadvantage is that we need to use something else for the active counter. An AtomicLong would be very suitable as it uses hardware assisted atomic operations (and therefore has no synchronization overhead).
If it is desirable to keep the firstUsage and lastUsege timestamps (I don't use them), these can lift along with the start time and be stored during the stop method. Note: this may be unacceptable if you need information on long running operations.

To test it out I hacked StopwatchImpl to return a StopwatchMeasurement from the start method. If you look at the implementation you see that StopwatchMeasurement implements Stopwatch so that the idiomatic usage still works:

Stopwatch simon = SimonManager.getStopwatch("simon.test").start(); // Do something simon.stop();

Of course this is a kludge that should be improved upon for a real implementation. In addition, I made no attempt to keep the active counter and the first/lastUsage timestamps.

Here are the results (again in ms per measurement):

Not bad! Although the results are quite dramatic for the higher thread counts, it still shaves off 20% in situations with less contention.

Recommendation: remove the thread local from StopwatchImpl and change the API as described above.

Replace synchronized HashMap with a ConcurrentHashMap in EnabledManager
The second thing I noticed is that getting a monitor from the SimonManager (the SimonManager.getStopwatch("simon.test") part) happens in a synchronized block and uses a plain HashMap as store. Since Java 5 there is a ConcurrentHashMap implementation which supposedly performs better in almost any circumstances.

Lets see. I remove some synchronized keywords, changed the map type, and made the code to add new monitors thread safe. This resulted in a modified EnabledManager.

Here are the results. The graph displays a few attempts that differ in the concurrency level parameter to the ConcurrentHashMap constructor. For more accuracy this graph displays the improvement in nanoseconds.

To my big surprise there is no measurable difference with modest contention. During high contention there is a reasonable positive effect that is suddenly converted to a negative effect for very high contention. Only when the currency level is set very high, does the effect stay positive.

Recommendation: don't change the manager.

Conclusions Although Simon is only young, it is off with a good start. Its clearly faster then Jamon and also on the documentation front Simon wins. Jamon has much more features so it is still a valid choice. When the recommendations in this article are followed, Simon could be better still.

Discussion
Follow the discussion of this evaluation on the Java Simon mailing list.

References
Code smells from Clean Code by Robin C. Martin:
[F3] Flag arguments
[N7] Names should describe side effects
[G14] Feature Envy
[G21] Understand the algorithm
[G25] Replace magic numbers with named constants

Code smells by me:
[CE6] Incomplete contract
[GE37] Unnecessary conversions

Monday, December 15, 2008

Don't use Intellij

IntelliJ is okay, but it has one bug (for more then 4 years) that absolutely drives me nuts: its steals focus, big time. And not just once either. It can steal the focus for at least 4 times within 30 seconds!

If are thinking about using IntelliJ: don't do it!

If you are already hooked up: please vote for this bug!

Saturday, December 6, 2008

Giving up on compiling a linux module

I recently build my new home server PC. The motherboard is a D94GCLF, a small mini-ITX (17x17cm) from intel with an Atom processor. Unfortunately, the on board ethernet card, a RTL8102EL is not supported by Ubuntu 8.10. The r8169 kernel module works but it drops most received packets so its not as fast as it should be.

Realtek does provide the source to build the module yourself. Unfortunately, the instructions are a bit sparse. Too say the least. After a 7 hour struggle, installing kernel and header sources, lots of googling, making symbolic links here and there I finally compiled a .ko file. Insmod refused to load it.

So I gave up.

Now my new server is up to full speed again. If you can't solve your problem in software, add hardware! I put in an old 100Mbit/s ethernet card, and all is well again.

Monday, November 17, 2008

Making Mac's open/save dialog fast again

Out of nowhere I had to wait 20 to 40 seconds on my year old iMac before I could open or save a file. Some people reported that iDisk could be the problem, but I don't even have a mac-account! However, those reports gave me a hint.

Indeed, a ls /Volumes/ showed this:

MacintoshHD Wuala ._Ipod_van_Corine
There it is! My girlfriend's IPod was not in sight, and I dumped Wuala because it stops working after a hibernate. A simple
sudo rm /Volumes/Wuala sudo rm /Volumes/._Ipod_van_Corine
and all was well again.

Thursday, November 13, 2008

Effective Wicket presentation

Yesterday was J-Fall, the fall conference of the NL-JUG. One of the presentations was my Effective Wicket talk. Feel free to download the slides of Effective Wicket (mirror) (PDF 4.1 Mb). If you want to see the notes skip to page 55.

If you want the original material (code/images etc.) for your own presentation, please drop me a note. Its all creative commons, copyright by JTeam.

Update 2009-01-07: Watch the presentation (Dutch spoken) on Vimeo: part 1 and part 2 (provided by bachelor-ict.nl).

Wednesday, October 15, 2008

Wicket extreme consistent URLs

Setting up a page to be behind a particular URL (aka mounting) in Wicket is fairly easy. Daan recently wrote REST like URLs for Wicket in which this is nicely explained. However, consistently keeping nice URLs, for example after a form submit, is a whole lot harder. So hard in fact, that even Wicket champion Igor believes it is currently not possible (it is an old post). His reasoning is of course completely correct, but he forgot one thing. Let me explain.

The problem
Many web applications have a login page. On it there is a form where you fill in your username and password. Suppose the page is mounted to:

http://example.com/login
with code like this in the application's init method:
mountBookmarkablePage("login", LoginPage.class);
When the user enters a wrong password, form validation (I assume you have a password validator) will fail and Wicket will redirect the user to a URL that points to the second version of the login page (the one with the error message). These URLs typically look like:
http://example.com/?wicket:interface=:0:::
For many applications this is just fine, in some its just too ugly.

A non solution
One of the proposed solutions is that you redirect to another page in the onError method of the form. E.g.

add(new Form(...) { @Override protected void onError() { setResponsePage(LoginPage.class); } }

The redirection will happen, and the URL is indeed that of the login page, but you will have no error messages. Instead of going to the second version of the login page, you have created a new instance of the login page!

Another attempt
We could pass the error code in the URL:

add(new Form(...) { @Override protected void onError() { PageParameters pp = new PageParameters(); pp.setString("error", "wronglogin"); setResponsePage(new LoginPage(pp)); } }

Unfortunately, with the default URL encoding stratey the URL will now be:
http://example.com/login/error/wronglogin
Not at all attractive.

Getting close
A fairly recent addition to Wicket is the HybridUrlCodingStrategy (and subclasses). Let's mount the login page with one of these:

mount(new HybridUrlCodingStrategy("login", LoginPage.class));

If a user now enters wrong credentials, Wicket will redirect you to
http://example.com/login.2
The .2 means the second version of the login page for the current session. If the user would delete the .2 from the URL, the HybridUrlCodingStrategy will find the latest available version of the page and serve that. A whole lot nicer but still not perfect.

The solution
If only if we could force Wicket to not display the version number. Well... we can! We'll have to do some coding first:

// First attempt, DO NOT USE public class NonVersionedHybridUrlCodingStrategy extends HybridUrlCodingStrategy { // ... trivial ctor @Override protected String addPageInfo( String url, PageInfo pageInfo) { // Do not add the version number as // super.addPageInfo would do. return url; } }

And now mount with:
mount(new NonVersionedHybridUrlCodingStrategy( "login", LoginPage.class));
Indeed, the version number is gone! However, there is still a tweak to be done. In some circumstance Wicket will redirect you to the latest version of the page, but now that redirect will be to the same URL. This is no problem for Firefox and IE, but Safari, Opera and many tools do not allow this. Here is the final version that does not have the problem:
/** * UrlCodingStrategy that will give the same * URL for every version of a page. * @author Erik van Oosten */ public class NonVersionedHybridUrlCodingStrategy extends HybridUrlCodingStrategy { public NonVersionedHybridUrlCodingStrategy( String mountPath, Class pageClass) { super(mountPath, pageClass, false); } @Override protected String addPageInfo( String url, PageInfo pageInfo) { // Do not add the version number as // super.addPageInfo would do. return url; } }

I named the URL strategy 'non versioned' as it no longer makes sense to have multiple versions of a page, just the last one will do. You should therefore also add the following fragment to each page constructor:
setVersioned(false);

Proof
Try it out! Go to tipSpot.com and try to login.

Monday, September 29, 2008

JAOO Observations

Here are my day 1 observations from the JAOO conference.

Languages evolve slowly. Despite an 100000 increase in storage capacity, 10000 increase in memory and 1000 increase in processing capacity in the last 10 years, programming languages look and feel the same as 10 years ago. The only real difference is that the language has become only a small part of what you need to learn. The major part is now in the libraries, which did grow by 1000 increase.
For new languages, it becomes harder to have your own infrastructure. That is why many are written on the JVM or the CLR.

If you are writing a social website, you must look at open social. The open social specification has been handed over to a foundation. It seems to have become really easy to incorporate it into your social website.

A long discussion on the beautiful (yep, that´s really the best word) presentation ´Designed as designer´ gave the following somewhat disappointing results:

  • ´Designed as designer´, or by substitution ´Work of art as designer´ is the wrong title. It is true that great works of art (including software projects) write themselves after the first draft, but the real topic of the presentation was that all works of art are produced by groups of people, not single individuals.
  • However, since we already operate in the group modus, our lives are not changed at all.

Wednesday, August 13, 2008

Java transaction boundary tricks

Controlling transactions is one of the fundamental things you must control well if your program is to be used by more then 1 user and has a database (as in about every web application). This article shows a little trick to get more out of transactions then most programs do. Since I am using Spring to demonstrate the principle, I will introduce Spring's transactions first. Feel free to skip that section if you know all about Spring transactions.

Using transaction from Spring
With Spring, controlling where a transaction is started can be as simple as adding the @Transactional annotation (and one extra line in the Spring configuration). The @Transactional is typically placed on the class that implements the interface of your service layer. For example you may have a MemberService:

public interface MemberService { List findMembers(MemberCriteria criteria); Member getMember(long id); Member updateMember(Member member); }
with the following implementation:
@Transactional public class DefaultMemberService implements MemberService { public List findMembers(MemberCriteria criteria) { ... } public Member getMember(long id) { ... } public Member updateMember(Member member) { ... } }
When you let Spring instantiate DefaultMemberService as a bean, all of its public methods will be automatically proxied in such a way that the method is executed within a transaction.

Java transactions are bound to a thread (think ThreadLocal), the proxy can therefore join an existing transaction in the current call stack, and by default that is what happens. As a result you can safely call other transactional service methods from within your own transaction methods.

Introducing the CommandService
Here is a kind of weird service that I came up with a couple of years ago, named after the Command pattern, the CommandService:

@Transactional public class DefaultCommandService implements CommandService { public void inTransaction(Runnable command) { command.run(); } public <T> T inTransaction(Callable<T> command) { try { return command.call(); } catch (Exception e) { throw new RuntimeException( "CommandService#inTransaction(" + command.toString() + ")", e); } } }
(Download link at bottom of article.)

As you can see it doesn't to anything except for executing the runnable or callable it is given. How can this be useful? Well, notice the @Transactional. Your block of code is now executed within a transaction!

Again, how can this be useful? Here are 3 use cases:

Use case 1: prevent fine grained service methods
Suppose you have a webpage that will update only one property of an entity, e.g. the email address of a member, and another page to update the password. Now to prevent concurrent update problems you want to get and update the member within the same transaction. You could make a new method on the service interface for each property you want to change. However, it is a good practice to keep interfaces concise. So instead we do something like this:

final long memberId = ...; final String newEmail = ...; Member freshMe = commandService.inTransaction(new Callable<Member>() { public Member call() throws Exception { Member freshMe = memberService.getMember(memberId); freshMe.setEmail(email.getNewEmail()); return memberService.updateMember(freshMe); } });
The member entity is retrieved and updated in the same transaction. This way we only need 2 methods in the member service interface to safely update any property of a member.

Use case 2: bundle operations for performance reasons
I once wrote some code that would keep thousands of LDAP records in sync with a relational database. In my test environment, a dodgy laptop, it would take 50 minutes to do the initial import of 15,000 records. Each record was persisted to the database in its own transaction. After a small code change, I used the CommandService to persist the records in groups of 20. What do you think happened? The total time to import these 15000 records dropped from 50 to 2 minutes! Not bad, 25 times faster through 4 lines of extra code.

Careful positioning of transaction boundaries can have a dramatic effect on performance. CommandService can help you do that.

Use case 3: transactions in unusual places
Let us look at a little bit more detailed example of the PeriodicRetriever from my previous article (changes in italic).

public class CachingPostalCodeService { private final Object postalCacheLock = new Object(); private List<PostalCode> postalCache; private PeriodicExecutor postalCacheReloader; private HibernatePostalCodeDao hibernatePostalCodeDao; public CachingPostalCodeService() { postalCacheReloader = new PeriodicExecutor( TimeUnit.MINUTES.toMillis(10), new Runnable() { public void run() { refreshPostalCache(); } public String toString() { return "Postal code cache reloader"; } }); } public List<PostalCode> getPostalCodes() { postalCacheReloader.requestStart(); synchronized (postalCacheLock) { return postalCache; } } private void refreshPostalCache() { List<PostalCode> newPostalCodes = Collections.unmodifiableList( hibernatePostalCodeDao.getAll() ); synchronized (postalCacheLock) { postalCache = newPostalCodes; } } // ... setter for hibernatePostalCodeDao ... }
If you try this, you may notice that the data is retrieved the first time, but not the second time! The exception message is clear too, something like "Hibernate requires a transaction". To understand this lets follow the two execution paths.

The first time getPostalCodes() is called, it is called through Spring's transaction proxy, so the method is executed within a transaction. So when the PeriodicRetriever is called, the runnable (which calls refreshPostalCache()) is immediately called within the same thread and thus also within the same transaction. No problem!

The second time getPostalCodes() is called (11 minutes later), PeriodicRetriever is called again but returns immediately. Meanwhile it starts a new thread which executes the runnable defined in CachingPostalCodeService's constructor. The important thing here is that the new thread does not join the transaction of its parent thread as transactions are bound to threads. The runnable then calls refreshPostalCache(). You may think you'll get a new transaction at that moment. However, a method call within the same class will never go through the transactional proxy. This is because the instance on which we call refreshPostalCache() is simply this, and not the proxy that was obtained from Spring. In other words: refreshPostalCache() is not executed within a transaction.

CommandService to the rescue (again, changes in italic):

public class CachingPostalCodeService { private final Object postalCacheLock = new Object(); private List<PostalCode> postalCache; private PeriodicExecutor postalCacheReloader; private HibernatePostalCodeDao hibernatePostalCodeDao; private CommandService commandService; public CachingPostalCodeService() { postalCacheReloader = new PeriodicExecutor( TimeUnit.MINUTES.toMillis(10), new Runnable() { public void run() { commandService.inTransaction(new Runnable() { public void run() { refreshPostalCache(); } }); } public String toString() { return "Postal code cache reloader"; } }); } // ... same as above ... // ... setters for hibernatePostalCodeDao and commandService }

Download
Feel free to use the complete version of CommandService in any way you see fit.

Conclusions
Watching your transaction boundaries can be very rewarding, both in code size as in performance. A command service, such as the one presented in this article can help you do that.

Wednesday, August 6, 2008

Asynchronous cache updates

This is the second article on Java concurrency. The first is Java thread control.

Many applications use more or less static reference data such as postal codes, exchange rates and externally stored text. Often this type of data can be safely cached in memory for performance reasons. To make sure that the cached data does not become stale for too long, it can be reloaded every 10 minutes, once a day or on any other schedule you might like.

Rather then creating custom code for every type of reference data, one can extract the hard concurrency code to a utility class. This article presents the PeriodicExecutor, a utility class I wrote years ago, and proved to be so useful and reliable that is has been traveling with me from project to project.

Here are the requirements I had in mind while writing PeriodicExecutor:

  1. Support any data retrieval and any cache mechanism.
  2. Refresh the cache in the background, do not slow down request handling.
  3. No idle threads, only start a thread when one is needed.
  4. Be resilient against errors, retry when they occur.
  5. Support flexible reloading schedules.
  6. Support initial loading of the memory cache.
Lets discuss these requirements and see how they are implemented.

1. Support any data retrieval and any cache mechanism.
Each type of data source needs its own retrieval code. E.g. JDBC code for database stuff and things like HttpClient for remote stuff. The cache implementation can also be different from case to case. For example in one case we just need to store a String, in another case we first transform the data to a Map. Therefore the utility class will only solve the the multi-threaded scheduling problem.

PeriodicExecutor solves this by letting the client code provide its reload task as a Runnable. The task is also responsible to make the results available when reloading is finished.

2. Refresh the cache in the background, do not slow down request handling.
When its time for a data reload, it does not make sense to delay normal request processing until the reload is done. Remember, we are caching for performance reasons! As a consequence PeriodicExecutor executes the client task in a separate thread.

PeriodicExecutor guarantees that the task is never run in parallel, but the client task must make sure that there are no concurrency problems the moment the cache is updated.

3. No idle threads, only start a thread when one is needed.
There are 2 approaches to thread handling. We can either start a thread upon initialization and keep that thread running all the time, or we can start a new thread each (short) time a reload is required. The first approach has the disadvantage that more resources are used then is necessary (though since Java 5 we could share an executor service with all reloaders). Inconvenient is that we need explicit code to shut down the thread (or executor service). Advantage of this approach is that a thread can schedule reloads precisely.

I chose the second approach: each time the cache is accessed by some code, that code must call method PeriodicExecutor#requestStart() to see if a reload is dictated by the schedule. If that is the case, the reload is started in a newly started background thread. A small disadvantage to this approach is that after a long period of inactivity, the first caller will trigger a reload, but until the reload is finished all cache users will see stale data.

4. Be resilient against errors, retry when they occur.
The reference data may be retrieved from an unreliable data source. When the reload fails, it should be tried again later. Normal processing can continue as the new background thread isolates it from exceptions. In addition, the reload task should only update the memory cache until after the reload succeeded.

PeriodicExecutor detects reload errors by catching exceptions thrown by the reload task. When an error occurred the next call to requestStart() will trigger a second reload attempt. After 2 failed attempts the next reload task is delayed for a couple of minutes to prevent trashing.

5. Support flexible reloading schedules.
PeriodicExecutor normally reloads with a fixed delay. However, it is easy to override this by providing your own next reload time. A future improvement would be to extract this code according to the strategy pattern.

6. Support initial loading of the memory cache.
Initially the memory cache is empty. As we are talking about reference data, it does not make sense to continue for most programs. PeriodicExecutor therefore also support synchronous loading and exception rethrowing. During a synchronous load the caller is blocked until the reload task is done. Before one task completed, the default is to run synchronously and rethrow exceptions from the reload task. Once one task completed, the default is to run asynchronously and to swallow exceptions (they are logged of course).

Example
Enough theory, how do you use it? Here is a simple example:

public class CachingPostalCodeService { private final Object postalCacheLock = new Object(); private List<PostalCode> postalCache; private PeriodicExecutor postalCacheReloader; public CachingPostalCodeService() { postalCacheReloader = new PeriodicExecutor( TimeUnit.MINUTES.toMillis(10), new Runnable() { public void run() { refreshPostalCache(); } public String toString() { return "Postal code cache reloader"; } }); } public List<PostalCode> getPostalCodes() { postalCacheReloader.requestStart(); synchronized (postalCacheLock) { return postalCache; } } private void refreshPostalCache() { List<PostalCode> newPostalCodes = .... // Don't forget to make newPostalCodes unmodifiable. synchronized (postalCacheLock) { postalCache = newPostalCodes; } } }
Notice that only a limited amount of code is needed to get all the discussed features. In this example a list of postal codes is cached in memory. The only user of the cache, method getPostalCodes, calls requestStart every time. Initially, when no data is in the cache, requestStart will block until the data has been loaded. After 10 minutes of use, the cache will be reloaded in the background. Note how the example synchronizes on postalCacheLock to prevent concurrency problems during the cache update.

Variations
Sometimes you only need to reload once a day, preferably early in the morning. The following example shows how one can override scheduled reload to 5 o'clock in the morning.

public CachingPostalCodeService() { postalCacheReloader = new PeriodicExecutor( 0, new Runnable() { // ...as above... }) { @Override protected long nextExecuteTime(long currentExecute) { return getNextTime(5); } }; } /** * @param hour the requested hour * @return the next time it is the given hour * in the JVM default time zone */ private static long getNextTime(int hour) { Calendar cal = Calendar.getInstance(); cal.set(Calendar.HOUR_OF_DAY, hour); cal.set(Calendar.MINUTE, 0); cal.set(Calendar.SECOND, 0); cal.set(Calendar.MILLISECOND, 0); if (cal.getTimeInMillis() < System.currentTimeMillis()) { cal.add(Calendar.DAY_OF_MONTH, 1); } return cal.getTimeInMillis(); }

If you need more control on when to execute synchronously or not, take a look at method PeriodicExecutor#requestStart(boolean).

Download
Download PeriodicExecutor. PeriodicExecutor can also be used with Java 1.4 (and perhaps 1.3 as well).

Other options
Probably the most complete FOS library for task scheduling is Quartz.

Enjoy!

Saturday, August 2, 2008

Java thread control

As promised in my book review of Effective java 2nd edition, here is an article that shows a neat trick to control threads.

The recommended way to stop a thread according to the book is to use a volatile boolean field as a flag to properly stop a running thread:

public class StopThread { private static volatile boolean stopRequested; public StopThread() { Thread t = new Thread(new Runnable() { public void run() { int i = 0; while (!stopRequested) i++; } }); t.start(); TimeUnit.SECONDS.sleep(1); stopRequested = true; } }
(See the book to understand why the volatile keyword is essential.) This is all nice, but this thread can only be started and stopped once. Here is a better implementation:
public class StopThread { private static volatile Thread currentThread; public StopThread() { Thread t = new Thread(new Runnable() { public void run() { int i = 0; while (currentThread == Thread.currentThread()) i++; } }); currentThread = t; t.start(); TimeUnit.SECONDS.sleep(1); currentThread = null; } }
By storing a reference to the running thread, the running thread itself can easily see whether the rest of the system also thinks it should be running. Restarting the thread becomes easy now. Just reference a new thread instance and the old thread will die.

The previous program just demonstrates the principle. A better starting point is the following program:

public class RestartableThread { private AtomicInteger threadNumber = new AtomicInteger(0); private volatile Thread currentThread; public RestartableThread() { startThread(); } private void startThread() { currentThread = new Thread( new RestartableRunnable(), "Restartable thread " + threadNumber.getAndIncrement()); currentThread.setUncaughtExceptionHandler(new Thread.UncaughtExceptionHandler() { public void uncaughtException(Thread t, Throwable e) { startThread(); logger.error(String.format("Thread '%s' has an uncaught exception and was restarted", t.getName()), e); } }); currentThread.start(); } public void shutdown() { currentThread = null; } private class RestartableRunnable implements Runnable { public void run() { while (true) { if (currentThread != Thread.currentThread()) { return; } try { // Do a bit of work. Make sure this takes a limited time. } catch (Exception ex) { logger.error("exception", ex); } } } } }

You can stop the thread by calling the shutdown() method. Failure to do so on program exit will prevent the JVM from stopping.
The runnable contains a try/catch to prevent the thread from dying. As a double fail safety mechanism, the uncaught exception handler is triggered for unhandled Throwables such as out of memory errors. The exception handler then simply starts a new thread so that execution continues. There are subtle advantages to this double approach. Exceptions are normally not so catastrophic, the thread could just continue. Out of memory errors however can be better dealt with by letting the thread terminate and make all its used memory available for the garbage collector.
If for any other reason you want to (re-)start the thread (for example because it is programmed to run only for a limited time), simply call startThread().

Make sure that the run condition is checked regularly. Consumer threads that wait on something like a queue should wake up regularly. For example, if you are waiting on new entries in a BlockingQueue this is better done with poll() then with get().

Conclusion
Despite the presence of the new and shiny executor services, it is still sometimes necessary to write your own thread. I hope that this little article will get you started on a robust yet flexible implementation.

Next thread article: a helper for asynchronous cache updates.

Friday, August 1, 2008

Effective Java 2nd edition - Book review

If you are even a bit serious about your java programming, this book is a must read. There is no other book that improved the quality of my java programming in such a dramatic way.

Now this was my opinion on the first edition. After having read the second edition, I can tell you that the same top quality is there, with lots and lots of big and tiny updates on top.

Ok, with that out of way, here are some comments:

  • In the introduction it says 'This book is not intended to be read from cover to cover'. Ignore that, read it from cover to cover anyway!
  • Initially I missed an overview of what changed since the 1st edition. But when I started reading I understood that such a list would be ridiculous. Every chapter has been improved in tiny, and sometimes big ways.
And now for some tiny counter noises:
  • Item 4: This item is about enforcing noninstantiability of classes that contain static methods only. However, except for saying that is nonsensical, it fails to provide a good reason for preventing this. I would say: don't bother. Let silly programmers instantiate the class. Firstly it can not do much wrong, secondly, any good IDE warns you about it.
  • Item 8: When overriding equals, one should use == instead of equals to compare fields of enum type. The item mentions that this should only be done for certain primitive fields.
  • Item 60: I disagree with the advice to throw NullPointerException in case an argument was null while the contract of the method specifies that this is not allowed. NPE is also thrown by the JVM and is therefore more suitable to indicate a programming error. IllegalArgumentException is in my opinion more suited to express that a method is used wrongly. In other words: a NPE indicates an error inside the method, an IAE indicates an error outside of the method.
  • Item 66: I recently saw a neater trick to control threads. More on this in the next post.
  • Item 68: One thing that still puzzles me is how one can write a consumer task with the new executor services. This item does not make this clear. Perhaps this use case still warrants the creation of your own thread.
Happy reading!

Update 2008-08-03: refined my critics on throwing a NPE vs IAE.

Thursday, July 24, 2008

Spring message resource weirdness

Today I got a bug report that e-mail subject appeared in Dutch while they should have been send in English. The first thing I check of course was whether the correct locale was used. It was.

Here is the situation:

  • Spring message source
  • 2 files: email-subjects.properties and email-subjects_nl.properties
  • requested locale: en_US
After a bit of debugging I noticed that Spring looks for the following files (in order):
  1. email-subjects_en_us.properties
  2. email-subjects_en.properties
  3. email-subjects_nl_NL.properties
  4. email-subjects_nl.properties
  5. email-subjects.properties
WTF?! It also looks for the default locale! Thank you Spring! This is really the first time you disappoint me (in 4 years, not bad). But of course I should have looked further, when will Spring ever let you down?

The solution? Well luckily the authors of the code realized this is not always what you want. An easy setFallbackToSystemLocale(false) overrides this funny behavior. I added <property name="fallbackToSystemLocale" value="false" /> to my config file and it all worked. Hurray for Spring!

Update 2008-08-04: adapted the article to the real solution.

Saturday, June 28, 2008

JPA and bit operations (Hibernate/Postgres)

Update: Don't use the code in this article as it is flawed. See the comments for more tips.

In my current data model one of the entities has a column of integer type that contains a large set of boolean flags. Of course this is well hidden in the domain model objects, so you can just call things like event.isCanceled().

However, in the administration interface we need to search for this entity based on separate flags. The JPA query I found in our code base is select ... where flagBits = 1 (canceled is expressed in bit 0). Oops. What if the entity is not only canceled, but also sold out (bit 1)? Quite wrong, lets fix it.

So, I rewrote the where clause to flagBits & 1 = 1 (& is the binary and operation in Postgres). Hibernate, our JPA implementation parses the query to do its ORM magic. Unfortunately, Hibernate does not understand the & operation (though I heard it did in Hibernate 2) so it will throw an exception.

Looking through the Hibernate and Postgres documentations I found the get_bit operation. It does not except integer types as their value, so a cast was needed: get_bit(''||flagBits, 1) == 1. Another exception. And hopelessly wrong as well, ''||flagBits creates a string with the decimal representation of the number, the function get_bit expects a byte array. Maybe: get_bit((''||flagBits)::bytea, 1) == 1. Not really, the problem is still there. And furthermore, Hibernate also does not support the || and :: operations, more exceptions from Hibernate. (The :: could be replaced by a supported cast operation.)

Back to basics. How can you access a given bit with more basic, supported operations? Fact: division by 2x will shift the bits right by x. Fact: a number modulo 2 gives 1 if bit 0 is set.

What follows is the where clause (flagBits / bitmask) % 2 = 1. Where bitmask is 1 for bit 0, and 2 for bit 1, etc. Oops, % is not supported either! Last attempt: mod(flagBits / bitmask, 2) = 1. Success at last.

Conclusion
  • JPA/Hibernate give an extensive but still limited subset of SQL expressiveness.
  • It pays off to pay attention during math classes.

Friday, June 13, 2008

Ubuntu server 7.10 to 8.04 upgrade experiences

Yesterday I upgraded my server at home (running 3 websites and a small e-mail server) from Ubuntu 7.10 to 8.04. My main system is a nice looking iMac, completed with an eeepc for traveling and kitchen table work. But I use Ubuntu in the closet and at work.
The upgrade went rather well. Of course, I had installed as little packages from outside the repositories as possible. I even did the upgrade over ssh while that is not recommended.

Unfortunately, I only just noticed that no e-mail was being received anymore. After some head scratching (I had also changed the firewall settings), I decided to compare my backup of /etc (yes, I had one!) to the current copy.

So how do you compare such a complex directory as /etc on a Mac? I tried Eclipse and IntelliJ. Despite their excellent compare tools, they have no such thing for complete directories. Searching the web for open source solutions did not give any conclusive evidence as well. Kdiff3 was mentioned most frequently, so I decided to try that one. Kdiff3 can be installed with MacPorts, but make sure you have plenty of time. It took more then more then 2 hours on this screaming iron!
Kdiff3 is not so intuitive as the commercial products Araxis and Beyond Compare (which I have used professionally), but once you get used to it, it is a nice and powerful merging tool.

And there it was: a small copy/paste error in my dovecot configuration. Postfix uses dovecot for authentication so every TLS connection failed. Copy, paste, restart, done!

Conclusions

  • Make sufficient backups of all your configuration files.
  • Use the packages from your distribution as much as possible. Do not install later versions of a package unless you are prepared to install it again and again and again.
  • Kdiff3 is an acceptable diff tool on the Mac.
  • Do not be afraid for an Ubuntu server upgrade!

Sunday, June 1, 2008

Do not offer money!

Money and open source, always an interesting combination. But how do we, simple developers (with no ambition but to make beautiful stuff and live from it), deal with this? Let me offer you my opinion.

Lets start with a real example on what you should not do. Here is a quote from an e-mail that was sent on the user list of well respected project xyz (which it is, is not important):

Hi!

I was wondering how I could make a contribution for xyz. I'm not talking about a code contribution but rather a small money contribution. I have got a lot of help here on this forum and in fact I don't think I have ever experienced this kind of help elsewhere!

Thanks to all that have helped and especially core xyz coders.

The author obviously has good intentions, so how can this be so disastrously wrong? Suppose you are celebrating Christmas (or any other traditional family event) at your mother-in-law´s house. She cooked really well and everything tasted just wonderfully. You stand up and say, "Mother, this was excellent. This must have been worth at least 400 euros. Here, this is for you." and you hand here 4 fresh 100 euros bills. You will probably understand that your mother-in-law will not be happy, and in fact she will likely not forget this for a long time. Will she have the same pleasure in cooking for you next year? I think not.

So what is the problem? As Dan Ariely, a professor of behavioral economics wrote in his excellent book Predictably Irrational (I read the Dutch translation "Waarom we altijd tijd te kort komen") there are 2 sets of norms, market norms and social norms. Market norms regulate how we interact with each other when money is involved. Social norms regulate all other interactions. So eating at a restaurant will fall under the market norms, eating at your mother-in-low falls under the social norms. It is both about eating, but if you think about it, there are vast differences in what you expect, how you deal with bad food, etc.

The coders of xyz do all of their hard work in their free time. They are not getting paid, they do this purely out of love for what they are creating, and perhaps because of the idea that it is useful to others, and perhaps because of the respect they will receive. Any social interaction therefore falls under the social norms. Now if you start to offer money, you will move the social interactions to market norms. Luckily the core coders of project xyz are wise and responded with a request to donate to a charity project. But suppose they accepted the money. Now why would they do their best next time when no money is involved? They might as well go drinking beer with friends. Or worse, why would non core contributors do their best if they do not get paid? If you know somebody else is getting paid for the same task, even if it only takes 1 minute, why would you spend any brain power on it?

What is even worse, after switching to market norms, it takes a lot of time before the social norms are reinstated. Here is quote from Eric Daugherty, author of the Java Email Server, on JES version 2.0:

I started down this path with passion and drive. After I'd completed the SMTP (I think) portion of the code base, I was contacted by Andrew Oliver about my interest in working on a JBoss mail server. He was working on a new project to build an enterprise class mail (and calender, etc) server build on JBoss (where he was employed at the time). This project, JBossMailServer at the time, seemed to eclipse what I was attempting, and given my already slowing momentum on the 2.0 branch, pretty much brought it to a screeching halt. I had a bit of interest in working on the JBoss version, but in truth some of the motivation for the projects that I work on is ego. Working on a project that wasn't 'mine' didn't quite have the same appeal, even if it was JBoss (remember, this was 4 years ago).

Apparently JBoss had found someone interested in the server, as they approached me with an offer to pay me for the task of completing the SMTP (or POP, I don't recall) functionality. Since I was a pretty easy task for me (I'd done it once already), and hey, they were paying me, I jumped on board. I worked through the deliverable they wanted and earned a little spending money.

However, a funny thing happened. Once I'd worked for pay on the project, it was really hard to get excited about working for free. Combine this with the lack of real ego payoff, and I drifted away from the project.

So when the previous state is restored (the money is gone), the market norms stay for quite a while. This is proven by Dan and colleagues and this is also Eric´s own observation. A troubling conclusion: by offering money for free services the result is less services.

Suppose you care for the success of an open source project, what can you do? My answer is simple: contribute by writing documentation, by answering questions, by writing patches, provide hosting and otherwise interact according to the social norms. If you are not able to do any of these things, it is better to stay out!

Acknowledgments
Thanks to Dan Ariely for his research. The Christmas dining example is from his book.

Sunday, May 4, 2008

Everything about Wicket internationalization

I have been using Wicket for about one and a half year (and followed it about a year longer), and only recently I realized the tremendous flexibility of Wicket. The core developers keep introducing major features by replacing large and small parts of the core, without changing much of the existing programming interface. It doesn't stop there though. Any good developer can do the same with relatively little effort. On occasion I replaced important parts of the core myself, just because I wanted it a little bit different.

Upcoming release 1.4 is all about introducing the long awaited generic models. But the release notes of Wicket 1.4-m1 contain another little gem. Created by non-core developer John Ray: child components within a wicket:message element. But before I explain what that means, lets first look at your internationalization options in Wicket.

This article requires you to have a very basic grasp of Wicket (what the application class is, how wicket:id is used, and simple components like Panel and Label). A bit of knowledge on Java properties helps as well.

Simple texts: wicket:message element, and wicket:message attribute
A simple web page easily contains dozens of texts that need i18n. It would be cumbersome to have to put a wicket:id on all of these, and then add a Label to each. Instead you can use the wicket:message element in the HTML file directly. For example in MyPanel.html:

<wicket:message key="helloworld">Hello</wicket:message>
The key (helloworld) is used to lookup the message in the property files. The default text (Hello) is used when the message could not be found.
The property files are (MyPanel.properties):
helloworld: Hello world

And for Dutch(MyPanel_nl.properties):
helloworld: Hallo wereld

The ouput with an English locale:
Hello world
You can do the same for HTML attributes. For example:
<input type="submit" value="Default text" wicket:message="value:helloworld"/>
Would result in:
<input type="submit" value="Hello world"/>
If there are multiple attributes to translate, add them prepended with a comma:
<input type="submit" value="Default text" wicket:message="value:helloworld,title:hellotitle"/>

Wicket's locale selection
As usual in Java i18n systems, messages are looked up by locale. The locale is automatically extracted from the HTTP request, but can also be explicitly set with getSession().setLocale(Locale.US).
If you need to override the locale for a specific component, override getLocale() on that component and it will be used by that component and its children.

If a client does not provide a preferred Locale, Java's default Locale is used. See Locale.getDefault() and Locale.setDefault(Locale)) for more information.

Finding the message
Another extremely powerful concept is how the message is looked up in the property files. Normally a properties file with the same name as the current component is used. E.g. if you are working within the component SummaryPanel, the message is looked up in the file SummaryPanel_nl_NL.properties (for Dutch, Netherlands locale). If that fails (either because the file does not exist, or because it does not contain the message), file SummaryPanel_nl.properties, and then SummaryPanel.properties are investigated. This is all just like Java's resource bundles work. But of course Wicket goes further. If the message is still not found, the property files of the parent class are investigated, and again, and again, all the way up to java.lang.Object. For each class all locale variants are searched for.

Components are reusable, but in order to make it really reusable you should be able to override the messages depending on where the component is used. This is facilitated by first looking up the message (following the algorithm above) for every parent in the component hierarchy (aka page hierarchy). Every component can override the messages of its child components, so the search starts at the page's properties and then trickles down to the component that uses it (yes, its top-down). In order to make overrides specific to a certain child component, you can prefix the message key with the component id of the child. See ComponentStringResourceLoader for more details.

If no message was found in the page hierarchy, another search starts which will look at your application class and its super classes. So Wicket first looks at MyApplication.properties (provided MyApplication is the name of your application) and then up the class hierarchy, passing org.apache.wicket.Application, up to java.lang.Object. This is how Wicket provides its many default i18n texts.

This might sound complicated, but in practice you simply have one properties file per page and some more for components that are reused over multiple pages. For smaller applications you can even put everything in one properties file. These rules work so well that you just do what you think is correct and it almost always just is.

One note on the location of the properties files: like HTML files, they must be in the same package (and same classloader) as the component they are associated with. In practice they live next to each other in the same directory.
If you want Wicket to get its resources from somewhere else (e.g. from a database), you can implement the interface org.apache.wicket.resource.loader.IStringResourceLoader and configure this in the init() method of your application class.

References: ExtensionResourceNameIterator, ComponentStringResourceLoader, and for real control freaks the new PackageStringResourceLoader.

Reloading and caching
When Wicket is started in development mode, changed properties files are detected and reloaded. To properly make use of this feature from an IDE, you should run Wicket directly from the compiled sources, for example with the Start file, included in every QuickStart. In Eclipse you just save the properties file, in IntelliJ you must do a make (Ctrl-F9) before changes are picked up. In production mode, resolved properties are heavily cached for performance. (Same applies to html files.)

Putting dynamic values in the messages
As soon as you need to add values to the messages, you also need to add some Java code. The java code provides the vales, but the rest of the text still comes from the properties file. One of the nice things here is how Wicket leverages java bean properties.

Here is a complete example. Not many frameworks make this so easy!

MyPanel.properties: summ: You, and ${otherCount} others, reviewed '${title}' \ and rated it ${rate}. MyPanel.html: <span wicket:id="summary">Text that will be replaced.</span> MyPanel.java: // Summary has getters for otherCount, title, etc. Summary summary = ...; add(new Label("summary", new StringResourceModel( "summ", this, new Model(summary))));
Resulting in something like:
<span>You, and 5 others, reviewed 'Wicket in Action' and rated it excellent.</span>
Property based message key It goes further: do you know a framework that can do this?
MyPanel.properties: summ.short: Thanks! summ.long: You, and ${otherCount} others, reviewed '${title}' \ and rated it ${rate}. Thanks! MyPanel.html: <span wicket:id="summary">Text that will be replaced.</span> MyPanel.java: // summary.getMsgPrefs().getStyle() returns "short" or "long" add(new Label("summary", new StringResourceModel( "summ.${msgPrefs.style}", this, new Model(summary))));
And more! The Java property syntax is also still available (e.g. like {0,Date}, {2,number,###.##} etc.)? You can find all forms in the javadocs of StringResourceModel.

The trouble with property files
Now this is all nice, but there is one problem you will frequently encountered while using string resources. In this problem you have a sentence that contains one or more dynamically constructed parts (for example some links). The order of these dynamic parts is potentially different for each locale. In the previous example, we may want to link to an information page on the title, and on the word 'others' open a modal window with a list of people.

If you try this with Wicket 1.3 (or any other Java framework I know of that uses resource bundles) you'll find that there is actually no pure way to do this. I know of 3 workarounds, but neither is very attractive. Lets review them. The first way is to split the texts before, after and between the dynamic components. This works good enough, but varying the order of the components is not possible unless you so something clever on the Java side, or have multiple HTML files. The latter is also workaround number 2: for each locale use a separate HTML file with the translation directly included (note: Wicket uses the same rules to lookup HTML files as it does for property files, e.g. MyPanel_nl.html just works like you expect it to). However, having more then one HTML file per component often leads to maintenance horror as changes must be synchronized over many files. The third workaround is to add placeholders to the message text that later are replaced by some HTML. That HTML is either hand written (in which case you may wonder why you started with Wicket at all), or you need to do some serious Wicket hacking to get components to render to a StringBuffer.

Now lets do this with Wicket 1.4-m1:

MyPanel.properties: summ: You, and ${othersLink}, reviewed ${titleLink} \ and rated it ${rate}. others: ${otherCount} others MyPanel.html: <wicket:message key="summ">Text that will be replaced. <span wicket:id="rate">rate</span> <a href="#" wicket:id="titleLink"> <span wicket:id="titleLabel">label</span></a> <a href="#" wicket:id="othersLink"> <span wicket:id="othersLabel">label<</span></a> </wicket:message> MyPanel.java: // Note, we directly add the embedded components // rate, othersLink and titleLink add(new Label("rate", new PropertyModel(summary, "rate"))); Link othLink = new Link("othersLink") { .... } add(othLink); othLink.add(new Label("othersLabel", new StringResourceModel( "others", this, new Model(summary)))); ExternalLink titleLink = new ExternalLink( "titleLink", summary.getTitleUrl()); add(titleLink); titleLink.add(new Label("titleLabel", new PropertyModel(summary, "title")));
The html file contains a wicket:message element with some embedded components. All text within the element is removed and replaced by the text from the properties file. Labels like ${othersLink} are replaced by the rendered component of the same wicket:id. That component must be embedded in the wicket:message element. Note that the order of the components is irrellevant. Also note that you will not see any reference to the wicket:message element in the java code.

If you want runnable code you can download the complete example code (a Maven 2 project based on Wicket's QuickStart). See below for instructions.

Getting string resources from code
Despite all of the above, there always remain some cases in which you need direct access to the messages. Luckily Wicket provides access through the Localizer. You can get the localizer from any component with getLocalizer().

Again, see the complete example code for an example.

Encoding troubles
Fairly unknown to beginning programmers is that you are only allowed to use ISO-8859-1 encoding in java properties files. If you live in Europe this is a fairly annoying as many languages have characters that are not known to ISO-8859-1 (for example the euro symbol €). The simple workaround is escaping: cree\u00EBr instead of creeër. (I always use this site to look up the ISO codepoint.)

But imagine you are making a site in Thai! Luckily Wicket can also read XML property files. Here is a fragment of the Thai properties that comes with Wicket:

<?xml version="1.0" encoding="UTF-8"?> <!DOCTYPE properties SYSTEM "http://java.sun.com/dtd/properties.dtd"> <properties> <entry key="Required">ข้อมูลใน ${label} เป็นที่ต้องการ.</entry> </properties>
Nice!

Sample code
To test the code in this article I created a small test application. Unzip it, and run mvn jetty:run to start it. When it is started access it on http://localhost:8080/i18ntest.

Changing resource settings
Finally, if you need more power, you can change all of Wicket's settings in the init() method of your application. Call getResourceSettings() to get a IResourceSettings instance. Let look at some of the options.

ThrowExceptionOnMissingResource: this will make Wicket throw an exception when a resource is missing. At first this may seem a convenient way to test that you listed all messages in a properties file. However, many standard Wicket components use defaults as fall back, so this option is mostly useless. Alternatively, watch for warnings in the log.

UseDefaultOnMissingResource: this will make Wicket use the default value (e.g. the text within the wicket:message element) when the resource is not found in a properties file. Setting this to true (the default) may hide errors for a long time, but setting this false will make your site not work if you made an error. Choose carefully.

There are many more options that allow you to change what, when and how properties are loaded. I have never found a use for these, but they are there if you need them.

Conclusion
As you have seen in this article, Wicket provides simple ways to do everything around internationalization, and some more less simple ways to completely customize this for the rare case you need it. Furthermore, the new Wicket 1.4 release will make it a lot better with support for componenta embedded in a wicket:message element. You can download the example to see everything in action.

Update 20080526: Incorperated the comment from Stefan Fußenegger on what happens if the user gives no preferred locale.

Update 20080527: Scott Swank put this article on the Wicket Wiki. Feel free to edit it there!

Update 20091203: Corrected text after Satish_j pointed out that I made a glaring mistake in the order messages are looked up in the page hierarchy.

Tuesday, April 15, 2008

Announcement: radiantpostfixextension release 1.0

I have just released a new extension for Radiant. From the description:
With this Radiant extension you can manage a small Postfix virtual domain. A Postfix virtual domain allows you to forward e-mail to another e-mail address. The extensions adds a tab to the admin screens of Radiant in which you to enter the redirects. It also supports crude but effective aliases and mail-lists. In addition the Postfix configuration is verified and instructions will be displayed if something is not right.
Here is an screendump to see the effect.

The first release, boldly named 1.0, should work under Radiant 0.6.3 and up.

Please visit the Radiant Postfix Extension project home.

Update 2008-04-21: just released version 1.1 as a file had the wrong name.

Wednesday, March 19, 2008

Installing Maven under Mac OSX

Update 2011-09-12: Forget everything that's below here. Just install HomeBrew and type brew install maven.

Update 2008-03-28: Instead of following this article, you are probably better of with MacPorts. Thanks for the suggestion Nino!

I did not know it could be this hard to install Maven. I've installed Maven a bunch of times already but never before on a Mac. It took me about 1 hour!

Here is what you need to do:

  • Download the apache-maven.tar.gz. If you follow the link on the maven download page both Firefox and Safari will get an html page but save it as a tar file. Look in the downloaded file (search for the string "apache-maven") and extract a real download link. When in doubt run md5 apache-maven-version.tar.gz on the result and compare it to the MD5 that is posted on the download page.
  • Extract the tar somewhere. For example with tar xzf apache-maven-version.tar.gz. Here it gives an error (tar: A lone zero block at 3083) but so far it does not seem to affect the result.
  • Ignore all the babbling about environment.plist files, and just create a ~/.bash_profile and put something like the following in it:
    export M2_HOME=/Users/erik/apps/apache-maven-2.0.8 export M2=$M2_HOME/bin export JAVA_HOME=/Library/Java/Home/ export PATH=$PATH:$M2
That's it. Good luck!

Wednesday, February 20, 2008

Repeated lesson: always distrust 3th party code generators

Warning: rant follows!

I am usually try to avoid generation tools. But I fell for it again. I thought that using Wicket Web Beans would speed up development of the administration screens in my current project. Indeed, the first screens were shown very quickly. But a few weeks later, it did not turn out to be so smooth as expected.

Here are some of my pain points:

  • You must explicitly exclude properties you do not want to see. So when you add the method boolean isPersisted() in the base class of all your entities, you must visit the meta data for each entity and add "-persisted".
  • Writing new types of input fields for specific properties works, but is a little clumsy.
  • I have no clue how to add validation. Especially when you consider the excellent vlidation support in Wicket has for this, it is kind weird that it is not clear how to do this with Wicket Web Beans.
  • When you have a recursive data structure, Wicket Web Beans crashes during the render phase with a StackOverflowException!
  • Somehow I failed to get custom input fields for collection properties.
Some positive points:
  • Support is good though the wwb user e-mail list. Unfortunately, I failed to provide the correct input (which is source code) as we changed the code constantly in an attempt to work around to shortcomings mentioned above. Secondly I did not want to sent our domain objects to a public list.
  • For simple domain models you are probably quickly set up and done.

Okay, we could have worked on Wicket Web Beans itself. After all it is open source. The code quality seems quite alright though I expect you need some time to get into it. Due to project dynamics and considering the excellent Wicket form support, I decided to pull out.

Unfortunately its now 2 days before the end of the sprint. So in 2 days we will shame ourselves in the demo before the customer with almost nothing working. Hopefully we can do better next week with regular Wicket forms.

Note: 'me too' comments are likely to be removed. I am also very interested in success stories!

Update, a couple of hours later: I reworded the article a bit as I found the first version too harsh.

Monday, January 21, 2008

I finally made up my mind

Those who read this blog more often may have noticed the minor change in the sub-title. It now reads:
Experiences from a hard core Java programmer that finally decided Ruby is a very very nice language, but rather stays on the JVM.
Compare it to the previous text:
Experiences from a hard core Java programmer that struggles with the question of switching to the world of Ruby and Ruby on Rails.
So how did I come to this conclusion? Here are some random considerations:
  • I have only once in the last 2 years used Ruby professionally (to do some counting in a site dump).
  • I do have 3 Ruby sites running (one Camping and two using Radiant). So I am not going to stop using Ruby anytime soon.
  • ActiveRecord totally kicks ass and is hugely impressive. Rails for the rest not so much (totally old school request response based, with like JSP code in the templates). It may be just my lack of experience, so don't base your opinion on this.
  • I am very addicted to correct refactoring and code completion.
  • As I wrote before, we now have Scala.
  • Update 2008-01-23: This one needs to be added. Since I recently became a father, it is simply too much to keep track of 2 worlds at the same time.
So if you expected me to be on amsterdam.rb right now, here is why.

GWT in action - Book review

I recently joined JTeam, a self declared GWT specialist. Of course I do not want to stay behind so I started reading GWT in Action.

That was a mistake. There is nothing wrong with GWT, I think it is a very nice component oriented technology in the spirit of Swing and Wicket, which makes it a lot easier to build very complex applications.

But back to the book. Reading the book is a lot like looking at an extremely knowledgeable professor. The kind of professors that know so much that they don't know where to begin. And like with the professor you will eventually get it if you hang on and are able to sort it all out in your head.
Well, I was fed up with it in chapter 5, half way through the 17 chapter book. Here are my mischiefs: 1) the text was constantly referring to chapters further away in the book, 2) it was continuously repeating itself (I had to read at least 4 times that it is okay to use something else then Eclipse), but worse, 3) the text was almost continuously on a different conceptual level then the reader could be. For example implementation details are mentioned (for example the event_bits attribute in the generated HTML) while they have nothing to do with making you understand the API (writing an event callback class). Or the other way around, at places where a simple examples would do wonders there is only some broad goal description.

Since there are many other books on GWT, you are probably better of with another one.