Thursday, November 16, 2006

SCXML as a Controller language for WEB applications ... A new MVC Platform and language independent Framework

I have been involved recently in the creation of a State Oriented Protocol Server that will serve my Company to control not only what but also when,and how the clients (other companies) can interact or use our services. Those clients bring users to services my Company offers.

After several evaluations of State Machine implementations and frameworks I found the apache commons-scxml[1] project which is an implementation of SCXML[2] in java. Based on it I was able to implement the whole architecture I had in my mind.

At the beginning I was just trying to get the "when". The "how" was just considered in part.

When: refers to the needed status (state) in which the server must be before allowing a client to use a given service. In other words my client can request this given service now because it is in a state as a unique result of being before in a previous one. From this state my client is allowed to use this service and maybe others but not all of them.

How: refers to the way the client presents the data to the final users.

I started using scxml for the "when" and partially for the "how". Partially because we use it just for encouraging clients to present certain legal text to customers for example. Still the clients are making flow decisions based on responses they get from us and this kind of makes me think a little bit more to come up with a better approach.

The other day I was following one of the commons-scxml mailing list threads where Rahul (commons-scxml author) pointed out a link [2] as part of a discussion about another issue. The important thing is that I read there "SCXML is a generic state-based controller for synchronising interaction events in any namespace not just VoiceXML or CCXML" and suddenly I just realized how to do the rest of the "how" part.

Clients are getting already from our responses the result of the given service and the state where the application is. They are using just the first part but they could be using the second one as well.

These two informations can make our clients implement the View part of an MVC architecture. In that way all business rules live in only one place and from that place the View to be presented is decided.

Clients can then just concentrate in developing the look and feel to offer the whole possible business usecases. They just have to know what the view is supposed to present the user and which are the possible next submissions the user will be able to do.

My Company owns the whole model and the controller. For the model my company needs java developers however for the controller part it just needs people aware of helper functions they can use and familiar with a scripting language (JEXL and JEL are currently supported but commons-scxml has a flexible plugin interface to put there your preferred one).

Having scxml as a controller language for web applications allows then to work using a proven design pattern like MVC in a framework that allows both separation of concerns and Rapid Application Development (RAD).

The concepts presented here have come from the implementation we have made in java but it does not mean they cannot be used in any other programming language. Using an scxml library like commons-scxml have great advantages:

*For the model you use your preferred OOP language (java has the advantage of communicating directly with commons-scxml but I am sure you can do the same from any other one).

*Your controller is ready to be used and it is language independent, it is easy to configure and you do not need a programmer to change a given business rule but you just need your programmer to expose the needed helper functions or methods from the model.

*For the view part the consumer (the client or the company itself) decide what language to use. If you are dealing with XML for responses as our project does, then allow me please to reccomend XSLT just because it is the natural language to make your transformations and making your final markup available independent on the browser capabilities. For purists I am not saying that XSLT cannot be combined with JSON+AJAX approaches to bring the user a nicer interface ;-)

More about scxml used in business cases can be found at [4].


User Agent Capabilities HTTP Service

When I first (1 year ago) got eXist [1] Native XML database and imported a plain un-fallback-ed WURFL [2] xml file I got a 200MB file that my eXist database was querying in my notebook (By that time Intel Pentium 3, 1 GB RAM) in about 0.02 sec just to give for example all the capabilities for a particular User Agent.

Several times I have been posting this idea in the wmlprogramming yahoo mailing list.

Currently I do not need WURFL other than to check my propietary database is good enough or not. In fact my Company needs some people involved in checking those capabilities in a daily basis. I am in charge of other tasks and cannot dedicate time to this idea if I do not find a sponsor. But the idea is actually pretty straightforward:

1)Get the WURFL file and you make a plain one without fallback.
2)Host the file in eXist Database.
3)Design the Queries you need (XQuery to get results, XUpdate to update data or insert new handsets).
4)No need for aditional development ... eXist has all you need GET/POST/REST/XML-RPC/SOAP and you use simple HTTP calls to request your capabilities.
5)Adding value:
A. To update your database you should periodically check wurfl with HEAD (not sure if it is working from the server it is hosted but it is a good idea to have that implemented). Recently someone was asking for an automatic way of checking updates to WURFL. Use HEAD HTTP Request for it!
B. When a new handset comes do not forget to get the UAProf from HTTP headers. Try to parse it automatically, try to discover any discrepancies and log the results. It will help people that are in charge of maintaining WURFL.

At the end of the row you end with a non language related service that your company can use to give the capabilities of a given UserAgent to all the implementations you have in C#, Python, Perl, Java, RoR etc.

Of course it would be ideal to have this as Open Source and one of the ideas I have been trying to sell is that a Company could host a service like this and provide free service based on IPs for some requests. If a portal has many visitors and a developer needs more throughput a fee could be charged. On the other hand the XQueries, XUpdates remain Open Source just in case someone wants to host his/her local copy of the XML database.



I will try to blog here my ideas and some of the work I do related to software development in general.

I am not a specific language programmer but rather I like to build architectures and select after that the best language to implement them.

The selection of the language will be affected by many things that I would summarize in three letters DRY.

Yes, Don't Repeat Yourself which I understand as Don't Repeat Your fellows.

So, if I have a requirement from a client to build a CRM why do I have to build the whole thing in Ruby On Rails just because I like to be in Vogue with the newest language? Or why if I have to build a whole CRM+ERP system I have to try to extend VTiger just because I think PHP is faster than Java and so I do not recognize the power of Compiere?

On the other hand of course there are times when I just happen to be controversial ... who does not? So, why do I have to build a new MVC having dozens of good ones out there? Yes I built my own one but still applying DRY because I simply assembled existing pieces.

Well I hope the information here will be useful at least for people that still are reiventing the wheel out there.