Saturday, May 31, 2014

Continuous delivery needs faster server startup. Could #Tomcat #Spring applications cope with that?

I am reluctant to accept th emyth that Java web applications don't fit well in agile environments. The main criticism is the fact that unless you use a commercial tool, a plugin architecture or an OSGI modularized app you will end up having to restart the server in order to deploy the latest application version.

But what about if actually the time that the application would take to load would be few seconds? Will a user differentiate 10 seconds delay originated from a slow database access or backend channel web service request in comparison with a server restart? The answer is: No, the user experiencing a delay does not really care about the nature of it. If we maintain a proper SLA this will be simply "a hiccup".

Even the fastest web services out there would be slow for certain random actions. You will never know if Gmail is taking longer because of a quick server restart for example. As long as you can get what you need a wait of few seconds won't natter.

If the definition of "Done" is "Deployed to production" developers will take more care of deployment performance. Waiting long time for a deployment, means disruption. Business will never approve minutes of downterm. On the other hand if you increase your WIP limits you will slow down, quality will suffer. This bottleneck becomes a great opportunity for improvement as expected. You have reach a Kaisen moment.

There is a need to work on deplopyment performence. Without addressing that important issue you will constantly delay deployments, you will constantly get more tasks piling up, the consequences will be terrible and will only be discovered if you are actually visualizing the value stream. You need to tune your server and your application for it to load faster. A restart should be a synonym of a snap.

In a typical Spring application you will proceed as with any other application. Logs are your friends. Go ahead and turn on debug level. Search for "initialization completed" and confirm how much time this process takes. In production you better use lazy initialization:
<beans ... default-lazy-init="true">
This contributes of course to the overall "Server startup". But there is more to do. Check this out.

It should become evident from simple log inspection what is the culprit for a slow server startup. Let us review the below example:

The first clear issue is that ”Initializing Spring root WebApplicationContext” takes 36 seconds which is almost half of the time the whole server takes to startup. The second issue is that “Initializing Spring FrameworkServlet” takes 14 seconds which is a moderate 10% of the whole server startup time. Spring tuning is needed in this example.

What about the other 40% of the time? Servers also can be tuned. For tomcat there is a lot we can do. For example, like explained in the link, if you find an entry for "SecureRandom" in catalina.out most likely your server is spending valuable seconds generating random patterns for use as session id. Using the below setting saves you those seconds as explained in the link:
I found myself saving ten seconds by adding the attribute and node shown below. The explanation again can be found in the provided link:
<web-app ... metadata-complete="true">
    <absolute-ordering />
Eliminating unnecessary jars demands to list them all first. Note that I am sorting them on purpose just in case we are using one in the app that is already included in the container or if we are using two versions of the same (which is impossible if you are using a plugin to check for suplicated anayway):
find /opt/tomcat/ -name "*.jar"|sed 's#.*/##'|sort
Then find which classes are inside the those you are unsure if you need or not. Now you do need the whole path so let us pick as an example jtidy-r938.jar which hibernate includes as dependency. Here are the relevant commands which you will need to adapt according to your machine paths:
find /opt/tomcat/ -name "jtidy*.jar"
jar -tvf /opt/tomcat/myapp/ROOT/WEB-INF/lib/jtidy-r8-20060801.jar
find /home/dev/workspace/ -name "*.java"|xargs grep "org.w3c.tidy"
In my case after saving 9MB worth of jar files I saw no startup time savings for the specific project I picked for this research.

I saw under 10 seconds savings after using the below in the app web.xml as suggested in the link:
<web-app ... metadata-complete="true">
        <absolute-ordering />
The use of the special attribute startStopThreads in server.xml#Engine should have no effect if you are running only one application however I saved some seconds I believe after I turned it on:
<Engine ... startStopThreads="0">

No comments: