Monday, June 28, 2010

Using Drools for Dynamic Data Transformations

In a previous post I showed how Velocity templates can be used to perform data transformations (Mappings). This time I will take the same sample to show how to use Drools to address the same problem.

I have written for that purpose a tutorial on Drools covering local environment configuration and debugging.

Drools project started as a Rules Engine but nowadays it includes way more than that. I am showing here just how to use Drools for a Rule Engine to apply configurable data transformations but Drools can be used in any portion of your software. After all, rules are needed everywhere.

Velocity is a template engine and so the authors have been very careful to stop the developer from affecting directly the domain model. At the same time it is not declarative but imperative and so it can become as hard to read/maintain as Java when it comes to complex rules usually coming with several if levels. In addition at the time of this writing there is a wonderful Eclipse plugin that allows stepping through the code of Drool files. These are some of the good things about Drools. Drools come with a learning curve though and as with any other powerful tool the risks of using it wrong are high.

One of the reasons you want to move to a Rules Engine approach is that you want to eliminate your "spaghetti code" resulting from several nested "if statements".

Wednesday, June 23, 2010

Velocity Templates for Dynamic Transformations

We can take advantage of Velocity templates to apply dynamic transformations and create any kind of format for example CSV files. This is useful to perform dynamic mappings while importing feeds coming from different clients.

The below example shows how we can produce a list of the money spent in tips (15%) for a list of restaurants and the lunch prices. The source data is injected (merged) into the velocity template and the destination data is generated from the velocity template after applying dynamic rules.

Using a similar approach you can transform let us say an Excel feed into a CSV file that has any number of columns distributed in any desired order, calculated or not.

Furthermore as the example shows you can get the final output from Velocity in Java Objects so you can perform additional tasks like validations.

The java code:

import java.util.HashMap;
import java.util.Map;
import java.util.List;
import java.util.ArrayList;

import org.apache.velocity.VelocityContext;

public class MapDemo {
public static void main(String[] args) throws Exception {

VelocityContext ctx = new VelocityContext();
ctx.put("math", new MathTool());

List inList = new ArrayList();
Map inMap = new HashMap();

inMap.put("a", "The Crab House");
inMap.put("b", "35.50");

inMap = new HashMap();
inMap.put("a", "Red Lobster");
inMap.put("b", "25.60");

ctx.put("inList", inList);

List outList = new ArrayList();
Map outMap = null;
List outHeaders = new ArrayList();

Writer writer = new StringWriter();
Velocity.mergeTemplate("./mapDemo.vm", ctx, writer );

String[] resultLines = writer.toString().split("[\\r\\n]");

int rowCount = 0;
for(String line : resultLines) {
if(line.trim().length() == 0){
String[] resultFields = line.split("\\|");
if(rowCount == 0){
for(String field : resultFields) {
//System.out.println("****" + field + "***");
if(resultFields.length != outHeaders.size()) {
throw new Exception("Line violates signature: '" + line + "'");
outMap = new HashMap();
int fieldCount = 0;
for(String field : resultFields) {
outMap.put(outHeaders.get(fieldCount), field.trim());
fieldCount ++;
rowCount ++;



The Velocity Template:

#foreach( $item in $inList )
$item.a|$item.b|$math.mul($item.b, .20)

How to run the program:
javac -cp velocity-1.6.4-dep.jar:velocity-tools-2.0.jar
java -cp .:velocity-1.6.4-dep.jar:velocity-tools-2.0.jar MapDemo

Sunday, June 20, 2010

Book Review: Liferay Portal 6 Enterprise Intranets

I have finished reading Liferay Portal 6 Enterprise Intranets

Overall a good book to make the reader understand the basic concept upon which Liferay is built as well as those out of the box capabilities it provides. Even though the book is not for Liferay developers they can beneficiate from knowing how Liferay is ultimately used by what I call Operations (Liferay administration).

The author covers the Portal-Group-Page-Portlet Liferay Portal organization, how to install and configure the portal, setting groups and permissions, creating pages and adding portlets to them.

Between the covered portlets are Asset Publisher, WIKI, Polls, Message Boards, Email Forms and Blogs.

It is explained how to use CMS and WCM capabilities to build pages from the portal UI.

It is shown how to extend Liferay functionality through the use of hooks (like "Documnent Library Record" which is as the author says "a special use of audit-hook") and custom fields(expando), SEO capabilities using sitemaps (, caching using ehCache, using velocity to render different content depending on User Agents, how Jasper Reports are integrated, how to agregate content inside a portlet (nested portlets, widgets, gadgets and the "Web Content Display" portlet).

An analysis about search optimization puts the reader in contact with Lucene, Solr and Open Search Framework. Search is for sure one of the areas where optimization and performance tuning are so necessary and it is important for the Liferay Admin to understand the possibilities to make the portal faster and so more scalable.

Thanks to the book I put my attention to some important considerations.

First email templates are still just so simple. I have written about this before, it is time to use a truly flexible template engine and Velocity is more than enough. At least in two real life projects I have successfully integrated Velocity templates to customize emails.

The second issue is related perhaps to the nature of the book which is oriented to non-developers but just people that want to get to speed deploying, configuring and customizing Liferay from the GUI. The book pushes for the use of ad-hoc pages built directly from the CMS/WCM and that might fit certain needs, hiwever in my opinion this is arguable the way to go as versioning, backup and disaster recovery could be affected if too many adhoc code is done directly from the portal UI. Following a proper SDLC would impose a more integrated versioning of the plugins source code and the UI developed markup. Certainly it looks promising that according to the author there is versioning support for those pages built from the UI. I would love to have time to test this in the near future.

Last but not least the book mentions JSR-286 however in reality the current portlets still do not use the new specification but rather stick to the JSR-168 specification. The Search Portlet is an example of it.

Monday, June 14, 2010

Downgrade Java after Ubuntu upgrade

I found myself so many times having to change Java version back to 1.5 after an Ubuntu upgrade/update that I decided to document it this time (a friend was facing the same issue)
sudo rm /usr/bin/java
sudo rm /usr/bin/javac
sudo ln -s /usr/local/jdk1.5.0_15/bin/java /usr/bin/java
sudo ln -s /usr/local/jdk1.5.0_15/bin/javac /usr/bin/javac