Don’t call your state ‘state’

In the OO world you are frowned upon if you call something object. Its time to extend this principle: don’t call your state state. This post is about why this is a bad idea and what you can do about it.

Recently we sat down to discuss the new data model for our messaging system at eBay’s Classifieds Group. One of the things we inherited from the past is the entity called conversation with a field called state. Possible values were Ok, On hold and Blocked.

So what was the problem?
A field called ‘state’ almost always has a very intuitive meaning. Unfortunately, the word is so vague that the meaning can easily warp, depending on the problem at hand. I noticed this in a couple of projects: the state field started to collect more and more possible values. With more values came increasingly difficult state transitions. This lead to code that was way more messy then necessary.

For example, in our conversation entity we could introduce the state Closed to indicate that a participant wants to stop the conversation. Then we continue by adding the state Archived to indicate that the conversation should be hidden until a new messages arrive.

What can we do?
The key observation is that each state value represents multiple behaviors. Think about it, what behavior is needed in each state? How do these behaviors change for each state? These questions will lead you to multiple fields that can represent the entire state of your entities.

Within a couple of minutes we found three behaviors we wanted to have for our conversations: a conversation is either visible or not (field visibility with values Displayed and Hidden), it will accept new messages or not (field acceptNew with values Accept and Reject) and we want to notify the recipient of a new message (or not) (field notifyOnNew with values Notify and Mute).Not only did our code become easier to extend and reason about, as a bonus we found a feature that would have been really hard with the old model: muting a conversation.

Conclusion
Don’t call your state ‘state’, instead, think about the behavior each state represents and model that instead.

Breaking the Circuit Breaker

The circuit breaker is this wonderful pattern to protect your application against resources that fail slowly. The idea is that you stop trying to use a resource when it has too many failures. Regular retries test the resource and will make the resource available again. The benefit is that your application can react quickly to a failed resource instead of hogging CPU, threads, network, etc. while you are waiting to find out the resource is unavailable.

So what's wrong?

Its the metaphor. In the classical description a circuit breaker has 3 states: the open state, the closed state and the half-open state. So what does it mean when the circuit breaker is open? When is a bridge open? When you can drive over it, or when you can sail through it? Only when you look at the first image you may see that a traditional open circuit breaker stops flow of electricity. To us that translates to no usage of the resource. In the 'closed' state electricity flows, which translates to having access to our resource. Now read that again and see if you can remember that!

Then we have a half-open state? Again, look at the first image. For such a switch half-open is still open. (A half-open bridge lets no traffic trough at all but that is another topic.) Why do we need the half-open state anyway? In the classical description we attempt to use the resource once while in this state. If it fails just once, we go back to the open state. This seems like a good idea, but let us think of modern networked applications. In such applications many requests are done simultaneously. So as soon as we switch to the half-open state for a retry, many, maybe hundreds of request will immediately try to use the resource, even if it is still down. This is exactly what we were trying to prevent!

Stop!

Although the circuit breaker is a great invention, I think we need a new metaphor, or at least some new terminology.

No more half-open

The first thing we can do is get rid of the half-open state. Instead, when its time to retry, we just let 1 client through to the resource. While that check is in progress we keep denying access to the resource for other clients; we stay in the same state. Only when the single check succeeds, we switch to the state in which we allow full access to the resource.

No more open

The second thing we need to do is to end the confusion on what it means to be 'open'. Instead I propose we call this state the broken state. No further explanation required. Good. In this state we do the regular retries.

Finally, to make things symmetric, I propose to rename the 'closed' state to flow state as all requests are granted.

Metaphor

Above I proposed new terminology but I failed to provide a new metaphor. Unfortunately metaphors are hard to find and too easy to get wrong. Perhaps a good metaphor should be related to the fact that we are limiting the number of errors we tolerate from a resource. If you have an idea, please let me know in a comment. I hope you liked my little rant. Any comments are always welcome.

—   ❧   —

Postscript: Sentries and the circuit breaker

The Sentries library contains a highly optimized circuit breaker implementation for Scala programs. The ideas in this article developed while writing Sentries. Feel free to have a look. As you can see there are only 2 states, the FlowState and the BrokenState. Note that the retryAt in BrokenState is a val; it can not be changed after initialization. When it is time to retry we replace the broken state with a new instance (in method attemptResetBrokenState).

Open source - why bother with anything else?

Since I work in a fine small company where we are breathing open source for at least a decade, it is sometimes weird to be confronted again by open source adversaries or agonists. For example, a colleague wrote a fine technical design based on Mule. All of a sudden we're asked to compare this to BEA AquaLogic and see whether we could implement the project with that. Now this is probably possible, and AquaLogic is probably a fine product family, but why bother?

Since there was already a Mule prototype, I found it a cumbersome idea. To get up to speed with AquaLogic you first have to find out what products in the AquaLogic family you need, and of course you only need a tiny bit of most of them. Secondly you have to go into a trajectory to get the software, including development licenses. Somehow money is not always a problem, but the time to get the products and start working always is, and the deadline won't move. Did I already mention I really hate bureaucracy?

Starting with open source often just takes 5 lines in a pom.xml and a few minutes of download time.

Okay, well suppose this was all taken care of and you are happily underway with development. You then run into a problem. Yes, you will, this is no different from open source. Now lets see how I typically deal with problems with using open source and see how this applies to closed source software.

Finding a solution to a problem with an open source product
1) (Re-)read the documentation
The first step is always to read the documentation. With the source jars attached in your favorite IDE, the javadoc is one key-press away.

2) Debugging
In step 2 we'll do some debugging. Again, with the source jars attached, tracing through your own code is as easy as tracing the open source code. I may not understand all the code, but I know what I need and I can read the JavaDoc of code encountered underway.
More often then not tracing leads to a deeper understanding of the used products, and I frequently find things that are useful for other parts of the project. The deeper understanding helps to form a solution. This can be changing your code, or patching the open source product. Otherwise it helps you formulate a more precise problem statement for the next steps.

3) Read more documentation
As I have now seen the code, I can search the available documentation more efficiently. So I do this first before going into the next step. Documentation in this phase can be anything, from manuals to blogs and forums.

4) Post questions
The last step is to post a question on a forum or mailing list. If you get this far, you are either lazy, you just missed something, or you found out that the library has a bug. The results of this step are not always satisfactory; it really depends on the community around the product. At least you can always change the open source product yourself.

Finding a solution to a problem with a closed source product

1) (Re-)read the documentation
Again, the first step is to read the documentation. However, javadoc is not always available. Rarely is it available in the form of a source jar.

2) Debugging
Oops, we can only trace our own code. Perhaps you can use de-compilation. Note that this might actually be illegal in your country (not in The Netherlands luckily). Secondly, the source might be obfuscated. And of course, I am not even talking about products that run as a complete separate program.

3) Read more documentation
As it is unlikely that debugging gave us more insight, we skip this step.

4) Post questions
In rare cases there are user communities around closed source. These are very valuable! However, usually you just have to ask the manufacturer. By lack of insight, the question won't be as detailed as with open source. And then we rely on the manufacturer. Some react quick and accurate, some don't even give you the the ability to ask questions. More often then not it costs lots of money and time. And of course meanwhile you will have to code workarounds yourself.

Conclusions

Despite the title I am not at all against closed and commercial software. Its just that as a programmer I find it hardly ever worth the bother.

WebBeans, the JSF cover up

At JavaPolis I saw very good presentation on WebBeans by Bob Lee. Bob did a very good job in explaining the concepts while thankfully still giving lots of code examples. WebBeans provides a way to use ordinary Java beans in the JSF environment. You do this with annotations.

Praise for WebBeans, and its main predecessor Seam; they really make developing JSF applications simpler. In particular, the conversation scope is a brilliant invention. But how simple does it get? Well lets see. For every aspect that can be managed there is an annotation. In one of the presentation examples I counted about 5 lines of ordinary Java code plus at least 12 annotations! Now there are ways to group annotations, but you do this by introducing more annotations! So instead of having to deal with JSF stuff, you now how to cook annotation soup.

I have always hated JSP programming, and now that I see that even talented people like Bob Lee and Gavin King are wrestling to fix JSF, I finally know why. The problem is that the whole idea of having contexts to store page data is flawed. It breaks all kinds of encapsulation rules and breaks the beloved type safety.

So what are the alternatives? Actually there are a few very nice web frameworks that do not use contexts. The one I am most familiar with is Wicket. Another often named contender is Tapestry, but of course there are many more. Wicket, like Swing, uses a component tree to build pages. Each component is completely responsible for its own markup (html) and data (the model). No encapsulation rules are broken. Another addition is GWT, not exactly a web framework, but nevertheless useful for making componentized web applications.

The demise of Java, long live the JVM!

The Java language is dying. Most people do not yet realize this yet, but it is inevitable. What do I mean by dying? Why do I think it is dying, and why is there lots of hope?

Dead languages
So what do I mean by dead? First of all, languages do not die just like that. There are still people programming in Cobol, and likewise, there are still people that can read old Greek fluently. But just like Cobol and every other programming language, Java will at some time be left to the dinosaur programmers. I think that Java is currently at its peak (or will soon be), and will go downwards from here. It will take some time, but the signs are there.

The signs
First of all, there is the rise of dynamic languages. They have been around for quite some time, but all of sudden there was Ruby on Rails. Many Java programmers bit the bullet and switched to be rewarded with a up to 10 times productivity boost. There are those that say this can be attributed to dynamic typing. I believe this is not the case. I think the boost is possible because the Java language is quite weak compared to Ruby; the same problem can be expressed in a lot less lines of Ruby then lines of Java. This has nothing to do with typing, but everything with how the language is structured.

Secondly, there is the way Java is extended over time. The extensions have been done with great care. This has paid off: Java is still mostly a clean and simple language. Even with the tiny changes, each new version brought its problems. The largest change so far has been the introduction of generics. I see generics as an improvement, but as I wrote earlier, they can be a pain to use.
And now there is the closures debate. Some people (like James Gosling) want to have them at every price. Others (like Joshua Bloch) say that the complexity budget has been used and that Java is not ready for big changes like closures. I agree with the latter.

Despite the care with which Java was and is being extended, the possibilities for doing so are reaching zero rapidly.

There is hope!
If there is one gem coming out of the Java world, it is the Java Virtual Machine. With each release the JVM has brought performance improvements. And this is likely to continue for the next versions. The JVM is still climbing and is nowhere near its peak. This can be proved by looking at the many languages that now run on the JVM. Everything from PHP, Lisp, Cobol to Ruby and Python. Some of these (in particular Ruby) are very well supported by several IDEs.

The Java language does not need closures or other more advanced stuff, there are many languages on the JVM that already provide those things: Ruby, Python, Lisp and Scala.

Yes, now there is Scala! Scala is a language for both the JVM and the CLR. Scala's syntax is as concise as Ruby's. It provides a very smooth transition from the Java language, but nevertheless is a complete functional language while at the same time staying purely object oriented and statically typed. And, as I learned at Javapolis, its library is already excellent, and even better: without any particular optimizations Scala programs outperform Java programs on the same JVM! What is not to like about Scala?

Disclaimer
Many statements in this article are based on personal observations and anecdotal evidence.

Update 2007-12-24: Despite the tone of this article, I am still writing Java and foresee to do so for quite a bit longer. And note: having fun doing it!

Mule patch accepted

Another product that accepted my patch. Till today I was proud to have patches in the following products:
- Xml2Db (specification for an aspect of the API, no code, long time ago)
- Struts (something with generating session ids, not long ago enough)
- Wicket (MixedUrlEncoding, other patches pending)
- Spring (a small documentation error)

Today this is extended with: - Mule (MailMessageAdapter potentially adds "null" string)

These were all small things, but I am still proud I was able to move open source further :)

For the future: I wrote a Camping application to manage a virtual e-mail domain for Postfix. My plan is to convert this to a Radiant extension and make it available under the MIT license.

SOA and Application/Integration databases

Hmm, I was going to write about a new insight. SOA has been reported to give huge cost savings. I figured that must have been because they used an application database and prevented the complexity of an integration database.

But of course, our hero Martin Fowler has already thought of that. Its right there in the second link.

If programming was as risky as parachute jumping...

Last Friday my company organized a parachute jump. Unfortunately the clouds were, typically for the season, too low and the wind was too hard. But I learned some interesting things anyway as the morning was spend on a crash course testing by Kees Blokland from PolTeq.

One of the major things for testing is risk analysis; the amount of time spend on testing an aspect of the system is proportional to the risk of that aspect (where risk is chance of failure times impact). For fun we made a risk analysis of parachute tandem jumping. To make it more real the instructor and owner of the jumping company also joined the discussion. He told us what he does to keep jumping safe and the risk low. He named two aspects 1) attitude, and 2) an open culture. An open culture is important because incidents must always be reported so that they can be analyzed and prevented in the future.

The attitude aspect coincided well with a discussion we had earlier about the tendency to not test well. Sometimes the client thinks it is not necessary, sometimes developers don't know how to test properly, and sometimes they don't want to.
I asked what the instructor did to maintain a good attitude so that we might apply his lessons in the world of software. Here was his answer:
1. parachute jumping in itself already attracts the right type of people,
2. everybody understands that safety and risk control is paramount,
3. maintain maximum openness about incidents,
4. make an example, even a bruised toe is reported to the authorities, and
5. take disciplinary measures only when there is an attitude problem. These measures are usually drastic, for example a withdrawal of the jumping permit.

So what can we learn from this? Here are my ideas derived from each answer:
1. Attract people with the right mind set. Look out for people that are both result and quality driven.
2. Train you project managers, and require them to organize risk assessments. In software not everything is deadly dangerous and not everythings needs to be tested.
3. Use bug metrics. I am not so sure about this one. But you should at least use automated testing.
4. Even a technical team leader can make bugs. Be sure that everybody knows that.
5. Don't blame people when they do something wrong. But give the pink slip when they refuse to improve.

Even when you will apply these lessons, you will have to realize that only when programming will be as risky as parachute jumping, programs will become bug free.

We made the jump 2 day later. It was great and there were no bugs.

What makes a programmer?

The answer to this rather philosophical question came up yesterday on the way home from my company's parachute jump event (more on this in a later post). My superior explained how he used to program in the past: "Try something, and if it works add more. Things that are used more then once are moved into methods." This approach will work for small projects, but will create a big mess over time. It will definitely create a mess immediately for larger projects. We are lucky that he is a good manager, but also lucky that he does not program for our customers.

So how does a good programmer work? In my opinion the sole work of a programmer is to map abstractions onto other abstractions, and to add new abstractions when this is not possible.

Lets give a simple example. XML documents are a well known abstraction for a hierarchical structure of labeled values. Files are a well known abstraction for a named sequence of bytes. To persist an XML document one writes code that maps the first abstraction to the second. How to do this is well described in the XML standard, but is definitely not the only way. I have written more performant mappings for specific cases.
A more complex example: to understand a voice command, one of the mappings might be from a sound sample to vowels and consonants.
Other types of abstractions are written in functional designs, test documents, etc.

So what makes an excellent programmer? An excellent programmer:
- Works with well known abstractions. There is not much to know about XML documents and Files. This makes them easy to understand and versatile. RDMSs provide an abstraction for storing tabular data. They are more complex, but well know.

- Will create small and consistent new abstractions.
An excellent programmer will create small levels of abstractions in his system with consistent terminology. For example, while processing sound samples it may be wise to create an intermediate level of abstraction that facilitates working with different sample rates and sample sizes. Consistency can be reached naming things carefully. Simply using the same names for the same things is a good step.

- Can work with many levels of abstraction at the same time.
At times it will be necessary to understand how the different abstraction mappings interact. For example, while persisting a phone book in an XML file, you need to know what each mapping does with character encoding.

So how do you do all this? Unfortunately I have no simple receipt as my manager has. I am afraid, excellent programming is still a craft.