Thursday, November 11, 2010

SFTP access only with OpenSSH

SFTP or secure FTP is preferred over plain FTP. If you just need to configure some users that have SFTP access only and you can live with a directory per user then you can use the method described below.

Nowadays most of the out-of-the-box pre-installed openssh versions will be greater than 4.9. For those setting up SFTP access is easy I have tested the below in Ubuntu 9.04 (Jaunty) with OOpenSSH_5.6p1, OpenSSL 0.9.8g.

The information below is the result of some research I have performed visiting a dozen of websites and trying things like RSSH and scponly. I found this to be the quickest and simple way to get SFTP working.
  1. Find out you are running version > 4.9
    $ ssh -v
  2. Create sftponly group
  3. $ groupadd sftponly
  4. Configure SFTP-access-only for group sftponly. Note I have commented out the ForceCommand line. In my Ubuntu with that line the server will authenticate the user but the user will get a "Connection closed" message right away.
    $ vi /etc/ssh/sshd_config
    #Subsystem sftp /usr/lib/openssh/sftp-server
    Subsystem sftp internal-sftp
    #The below must terminate the file
    Match Group sftponly
        ChrootDirectory %h
        AllowTCPForwarding no
        X11Forwarding no
        ForceCommand internal-sftp
  5. Add a user for example "report". The home directory should be /home/ and the shell must be set to a false shell.
    $ useradd -d /home/report -s /bin/false -m report
  6. Alternatively modify an existing user
    $ usermod -d /home/report -s /bin/false report
  7. Assign user to group
    $ usermod -g sftponly report
  8. Assign a password to the user
    $ passwd report
  9. Modify ownership and permissions to the home directory
    $ chown root:root /home/report
    $ chmod 755 /home/report
  10. Create a folder and assign permissions for the user. Within this folder the user will be able to add/remove folders and files. Of course permissions can vary depending on what you want to achieve.
    $ mkdir /home/report/reports
    $ chown report:report /home/report/reports
    $ chmod 755 /home/report/reports
  11. New users. Just repeat steps 4-9.

If you liked this then it is time for you to check how to simplify sftp user creation.

Some useful resources

Tuesday, November 09, 2010

IReport No suitable driver found

I knew I had checked for everything and still some of my Jasper reports were unable to run from Jasper:
Caused by: java.sql.SQLException: No suitable driver found for jdbc:...

In my case it turned out to be a class loader problem. For some reason (custom class loaders?) the Netbeans GUI is not able to find the JDBC driver. Interesting enough this does not happen for all reports and probably there are situations (specific xml tags inside jrxml?) where the bug shows up.

Some proposed solutions

Sometimes it just works copying your diver jar file into the iReport lib folder. Below are examples in OSX and Windows for my version of iReport (Your folders might differ slightly, that is why I am providing a folder list). The specific driver failing in my case was the sqlite driver.

In Mac OSX
Nestor-Urquizas-MacBook-Pro:~ nestor$ ls -l /Applications/ 
total 7240
-rw-r--r--  1 nestor  admin   255928 Jul 20 11:29 boot.jar
-rwxr-xr-x  1 nestor  admin    14731 Jul 20 11:29 nbexec
-rw-r--r--  1 nestor  admin    93696 Jul 20 11:29 nbexec.exe
-rw-r--r--  1 nestor  admin    23345 Jul 20 11:29 org-openide-modules.jar
-rw-r--r--  1 nestor  admin   626203 Jul 20 11:29 org-openide-util.jar
-rw-r--r--  1 nestor  admin  2684154 Oct 28 20:32 sqlitejdbc-

In Windows
C:\>dir "C:\Program Files\Jaspersoft\iReport-3.7.4\platform9\lib"
 Volume in drive C has no label.
 Volume Serial Number is DC32-F12F

 Directory of C:\Program Files\Jaspersoft\iReport-3.7.4\platform9\lib

11/09/2010  03:03 PM    <DIR>          .
11/09/2010  03:03 PM    <DIR>          ..
07/20/2010  10:37 AM           255,928 boot.jar
07/20/2010  10:37 AM            14,731 nbexec
07/20/2010  10:37 AM            97,792 nbexec.exe
07/20/2010  10:37 AM            93,696 nbexec.exe_original
07/20/2010  10:37 AM            23,345 org-openide-modules.jar
07/20/2010  10:37 AM           626,203 org-openide-util.jar
10/28/2010  07:32 PM         2,684,154 sqlitejdbc-
But sometimes it does not. Some people (especially on Widnows and linux) report dropping the jar in the $JAVA_HOME/lib/ext/

A perhaps better solution

Why these hacks? What is really going on?

The answer to this question will need to be found in the specific environment and the way iReport (netbeans) work. If you are running a Unix/Linux/OSX syste you can use lsof to find out what is going on and get the issue resolved.

My test case would be a report which uses for example sqlserver and a subreport which uses sqlite. You might notice that the subreport cannot render and sometimes iReport will complain about no suitable driver while others simply the report comes back blank (which is even worst we agree). Follow the below steps and let me know if it does work for you.
  1. Remove from preferences classpath any driver jar file
  2. Run the master report
  3. Check with "lsof|grep sqlite" that ireport did not open a sqlite file (you need to look for the java entries or you could use option -p to get a faster response just for the ireport process). If it lists any sqlite file open just delete the file. In OSX iReport opens a temp file like /private/var/folders/_6/_1d763x107sffygx2l8rr3vw0000gn/T/libsqlitejdbc-1619728097364591810.lib so be aware you are not looking only for jar extension. Even though that file is temporary it might have a caching impact. Sometimes I could even see previous classpath definitions that were already removed which is clearly coming from some internal caching. It is very important that lsof shows no sqlite files open and that you get an error close to the below:
    java.sql.SQLException: No suitable driver found for jdbc:sqlite:/path/to/mysql.db
  4. Now that we know we have a system with no caching restart ireport
  5. Go to preferences and in the classpath add the jar file
  6. The result of lsof command will be showing the newly added jar and then you think you got it: if the jar is referenced iReport should have it in the classpath indeed. Not so fast my friend, classloaders work in misterious ways ;-) Unfortunately it will fail again.
  7. Delete the main report jasper file and restart iReport.
  8. Edit the main report with something like:
  9. Now when you run it you get the driver loaded indeed. What did we do? We just forced the loading of the driver needed by the subreport from the main report. Note that in OSX as I said before you get something like /private/var/folders/_6/_1d763x107sffygx2l8rr3vw0000gn/T/sqlite-3.7.2-libsqlitejdbc.jnilib from lsof instead of the jar driver. Optimizations? Who knows but that caching game really takes the life out of any mortal

Monday, November 08, 2010

XSS and CSRF protection in Spring MVC Framework

UPDATE: Use UUID.randomUUID().toString() instead of trying to build the token yourself. No need to explain what these vulnerabilities are about. As developers it is important to understand them and to protect the software we create against them.

At the time of this writing Spring MVC Framework is still not providing an out-of-the-box protection

XSS protection can be achieved sanitizing the requests and the responses. In Java world you use a filter for the first. For the second XML must be escaped. JSTL has broad support for it as Spring tags has. If you are using scriptlets be sure to come up with your own ecaping mechanism (I love scriptlets but certainly you are safer using taglibs. Beware if you use taglibs to sanitize your resulting-from-request-responses.

There are several samples of XSS filters out there. I have used some variations of Stripes XSSFilter in the past. There are also several libraries that provide XSS protection but they will commonly push for you to use their own taglibs. I prefer to do it myself while I wait for native Spring support.

CSRF protection can be achieved (or at least mitigated) through the use of the Synchronizer Token Pattern. While again a filter or even a listener could be used I prefer to rely on simple Controller logic. In reality I prefer to protect my BHUB single entry points.

So I have a RootController from where I inherit (I am excluding my internal wrappers around getting and setting in session and request attributes but the below should be self explanatory):

public class RootController {
    protected void init(ControllerContext ctx) {
        ctx.setRequestAttribute("module", ctx.getModuleNameFromCurrentUrl());
        setAdvancedSearchAvailable(ctx, false);

    private void initializeCsfrToken(ControllerContext ctx) {
       String csrfToken = ctx.getSessionAttribute(ControllerContext.CSRF_TOKEN, "");
       if(Utils.isEmpty(csrfToken)) {
           ctx.setSessionAttribute(ctx, ControllerContext.CSRF_TOKEN, generateCsrfToken(ctx));
       ctx.setRequestAttribute(ControllerContext.CSRF_TOKEN, csrfToken);
    private String generateCsrfToken(ControllerContext ctx) {
        //long seed = System.currentTimeMillis(); 
        //Random r = new Random();
        //return Long.toString(seed) + Long.toString(Math.abs(r.nextLong()));
        return java.util.UUID.randomUUID.toString();
    protected boolean isValidCsrfToken(ControllerContext ctx) {
        String csrfParamToken = ctx.getParameter(ControllerContext.CSRF_TOKEN);
        String csrfSessionToken = ctx.getSessionAttribute(ControllerContext.CSRF_TOKEN, "");
        if(!Utils.isEmpty(csrfParamToken) && !Utils.isEmpty(csrfSessionToken) && csrfParamToken.equals(csrfSessionToken)) {
            return true;
        } else {
            //Log this as this can be a security threat
            Log.warn("Invalid security Token. Supplied token: " + csrfParamToken + ". Session token: " + csrfSessionToken + ". IP: " + ctx.request.getRemoteAddr());
            return false;

From individual Controllers methods (I am excluding my wrapper around applicationContext#getMessage()) we have:
public ModelAndView list(HttpServletRequest request,
           HttpServletResponse response,
           //This is a hack just to provide an uniform way to present global errors in the front end. @ModelAttribute should never be used in this method
           @ModelAttribute("employee") Employee employee,
           BindingResult result) throws ServletException, IOException {

       ControllerContext ctx = new ControllerContext(request, response);
       if (!isValidCsrfToken(ctx)) {
           result.addError(new ObjectError("employee", getMessage("error.invalidCsrfToken")));
           return getModelAndView(ctx, "employee/list");

Some JSP samples:
<div id="global_error"><form:errors path="employee"
             cssClass="errors" /></div>
<input type="hidden" name="ctoken" id="ctoken" value="${ctoken}"/>
<li><a href="<spring:url value="/employee/list?ctoken=${ctoken}"/>" class="navLink"><span>Employee</span></a></li>

In file (for internationalization):
error.invalidCsrfToken=Invalid Security Token!

Note that this solution creates a token for the whole session duration. I do not like breaking the back button functionality so I take especial care when submitting so the user lands in a different than submission-page. In addition the ctoken is gotten in JSP from an attribute instead of a param. It is assumed that the param was sanitized before.

Again it would be ideal to have this protection provided out of the box by Spring. A combination of the form tag with annotations in a per Controller method basis (with the help of AOP) is certainly possible. While I wait for that I keep on going the old school way instead of adding just another framework to protect my apps from CSRF attacks.
Finally it is better to use a header instead of a param in the case you must use GET protection. Of course this demands an AJAX driven app (AKA SPA).

Friday, November 05, 2010

Data Encryption with Jasypt Spring JPA and Hibernate

If you are storing private user data you must encrypt it to protect your users and also to avoid consequences (see HIPAA and California/Massachusetts Privacy Laws as an example).

Before continuing let me state that this is not a post about if you should store private data or not, that depends on the real necessities and your professional judgment. This is not a post about how to store your keyring the right way neither a security-for-pros post.

Jasypt is simple and yet tested by security experts that do the job for you so do not rely on obscurity to protect your data, use a Library like Jasypt.

Spring allows to clean your code especially if you use annotations and JPA facilitates persistance supporting annotations. JPA unfortunately is missing still so many features that I forgot a while ago about changing my persistence provider in the future. Hibernate provides a really clean (at least in comparison with others I had to implement in the past) solution.

I am sharing a typical configuration to get a JUnit test up and running but exactly the same applies to a container managed (Web) application.
So in your pom.xml
<!-- Encryption / Decryption -->

In spring config file:
<context:property-placeholder location="" ignore-unresolvable="true"/>
<bean id="hibernateStringEncryptor" class="org.jasypt.hibernate.encryptor.HibernatePBEStringEncryptor">
        <property name="registeredName">
        <property name="password">

In your (Which of course will use a different password than the one you use in development/integration, staging and production environments ):
#Holder for all general test properties

In JPA Entities the proposed annotation on top of the getter did not work for me. I had to annotate the field itself which makes sense BTW:

private String ssn;   

Monday, November 01, 2010

Acceptance Test Driven Development ATDD with Selenium

The User Interface defines how the application works. Automated testing from the user interface is therefore a must-do. And since Front End Engineers are in charge of developing the user interface the best investment you can do for Quality Assurance ( QA ) is to have Front End Engineers responsible for testing the application.

No need to mention how important is to have automated tests as part of any SDLC. Unfortunately for many projects it is prohibited to do any kind of Test Driven Development (TDD). Whatever the reason is (and yes, there are reasons that are sometimes beyond the IT team control) still Tests are necessary.

The tests you cannot forget about are those resulting from existing bugs (Test Driven Bug Fix or TDBF). When you correct something you should write a test that asserts that part of the application works as expected. The reason is that you want to be sure the very same problem does not hit you back. It is embarrassing to hear from Business side "This problem is happening again", period.

So you might not be able to afford TDD but at least you should be able to provide Development with Automated Tests (Let us call it DAT as these days we are supposed to remember abbreviations for almost anything :-)

While U-TDD (Unit TDD) is in reality invisible for the stake holders A-TDD (Acceptance TDD) is not.

User Stories have proven to be an effective mechanism to allow both, business and developers talk the same language. Plain English constructs "Given, When, Then" are a good starting point to build automated acceptance tests. There are multiple testing frameworks supporting code assertions for user stories. Selenium is one that I think is leading the race.

Your Test team (also called QA) can use any language regardless of the language being used for your real website. The reality is that small and medium size companies commonly cannot afford paying for dedicated people for this matter (Even though this is the ideal way to go either for Agile or Waterfall approaches). But as an experienced software architect you know that separation of concerns is the most important principle of successful enterprise software development so even if you decide to use the same language and keep it maintained by the same team of programmers you should:

1. Convince stake holders they must agree on User Stories.
2. Put the acceptance tests that assert the user stories in an acceptance tests project (Do not use your application project to host acceptance tests).
3. Externalize any configurations as URLs and credentials.
4. Schedule (at least daily) automated acceptance tests. Continuum Integration is a must-have. I have built it myself in the past but do not waste your time there are really good tools in the market for this.

The development team must be focused on what the customer really wants. Your customer (stake holders) must be in control to evaluate if the software you are building meets the requirements. There is then, as I have written before, a need for business people to write effective user stories.

Here is how the User Story 1 should be written by our stake holders. Note that we could divide this User Story in more than just one but for the purpose of showing the use of differenct methods in the final Java class I prefer to include all related scenarios:
User Story 1:
Given a user is not logged in 
When any page is hit
Then the user should be asked to authenticate

Given a user with admin role is logged in 
When an admin page is hit
Then the user should see the page

Given a user with just regular user role is logged in 
When an admin page is hit
Then the user should see an error page

In the Java world I prefer Maven as building tool. Below you can find what a POM file will look like to use Selenium 2 for Acceptance tests.

<project xmlns="" xmlns:xsi=""
    <description>Acceptance tests for my web application</description>

Below is an example of a properties file to hold the target application URL, the WebDriver to use (HTMLUnit driver in this case) and user names and passwords to test:

Here is a class you can use to get the properties from the file above in you acceptance tests:
package com.nestorurquiza.acceptance;

import java.util.Properties;

import org.openqa.selenium.WebDriver;
import org.openqa.selenium.htmlunit.HtmlUnitDriver;

 * A VM argument providing the properties path parameter is needed.
 * for example:
 * Here is an example of such a property file
* #acceptanceEnvironment.propeties
 * baseUrl=http://localhost:8080
 * webDriverClass=org.openqa.selenium.htmlunit.HtmlUnitDriver
 * adminUserPassword=test
 * regularUserPassword=test
* */ public class EnvironmentProperties { private static final String ACCEPTANCE_ENVIRONMENT_PROPERTIES = ""; private static final EnvironmentProperties INSTANCE = new EnvironmentProperties(); private static Properties properties = null; private EnvironmentProperties() { } public static Properties getInstance() throws FileNotFoundException { if(properties == null) { properties = new Properties(); //Use String acceptanceEnvironmentPropertiesPath = System.getProperty(ACCEPTANCE_ENVIRONMENT_PROPERTIES); if(acceptanceEnvironmentPropertiesPath != null && acceptanceEnvironmentPropertiesPath.trim().length() > 0){ InputStream in = new FileInputStream(acceptanceEnvironmentPropertiesPath); if(in != null) { try { properties.load(in); in.close(); } catch (IOException e) { e.printStackTrace(); } } } } return properties; } public static WebDriver getWebDriver() throws FileNotFoundException { //The simplest not environment specific Driver WebDriver webDriver = new HtmlUnitDriver(); Class c; try { c = Class.forName(EnvironmentProperties.getInstance().getProperty("webDriverClass")); webDriver = (WebDriver) c.newInstance(); } catch (ClassNotFoundException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (InstantiationException e) { // TODO Auto-generated catch block e.printStackTrace(); } catch (IllegalAccessException e) { // TODO Auto-generated catch block e.printStackTrace(); } return webDriver; } public static String getBaseUrl() throws FileNotFoundException { return EnvironmentProperties.getInstance().getProperty("baseUrl"); } public static String getAdminUserName() throws FileNotFoundException { return EnvironmentProperties.getInstance().getProperty("adminUserName"); } public static String getAdminUserPassword() throws FileNotFoundException { return EnvironmentProperties.getInstance().getProperty("adminUserPassword"); } public static String getRegularUserName() throws FileNotFoundException { return EnvironmentProperties.getInstance().getProperty("regularUserName"); } public static String getRegularUserPassword() throws FileNotFoundException { return EnvironmentProperties.getInstance().getProperty("regularUserPassword"); } }

Finally here is a sample acceptance test that makes several assertions, all of them included in our User Story 1. Note that the test below asserts a site protected with Spring Security:
package com.nestorurquiza.acceptance;

import static org.hamcrest.CoreMatchers.*;
import static org.junit.Assert.assertThat;


import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import org.openqa.selenium.By;
import org.openqa.selenium.WebDriver;
import org.openqa.selenium.WebElement;

 * Verifying behavior of UserStory1.
public class UserStory1Test {

    private WebDriver driver = null;
    public void aTearUpMethodForEachTest() throws FileNotFoundException {
        driver = EnvironmentProperties.getWebDriver();

    public void aTearDownMethodForEachTest() {

    public void givenNotLoggedInWhenAnyPageIsHitThenShouldBeAskedToAuthenticate() throws FileNotFoundException {

        String url = EnvironmentProperties.getBaseUrl() + "/client/list";
        assertThat(driver.findElement("loginForm")).getAttribute("action"), is("j_spring_security_check"));
    public void givenLoggedInAndAdminUserRoleWhenAnAdminPageIsHitThenShouldSeeThePage() throws FileNotFoundException {
        String url = EnvironmentProperties.getBaseUrl() + "/";
        WebElement loginForm = driver.findElement("loginForm"));
        WebElement userName = loginForm.findElement("j_username"));
        WebElement password = loginForm.findElement("j_password"));
        url = EnvironmentProperties.getBaseUrl() + "/client/list";
        //Not supported by HtmlUnit
        //assertThat(driver.findElement(By.cssSelector(".form-navigation")), is(notNullValue()));
        assertThat(driver.findElement(By.className("form-navigation")), is(notNullValue()));
    public void givenLoggedInAndRegularUserRoleWhenAnAdminPageIsHitThenShouldSeeAnErrorPage() throws FileNotFoundException {
        String url = EnvironmentProperties.getBaseUrl() + "/";
        WebElement loginForm = driver.findElement("loginForm"));
        WebElement userName = loginForm.findElement("j_username"));
        WebElement password = loginForm.findElement("j_password"));
        url = EnvironmentProperties.getBaseUrl() + "/client/list";
        //Not supported by HtmlUnit
        //assertThat(driver.findElement(By.cssSelector(".form-navigation")), is(nullValue()));
        assertThat(driver.findElement(By.className("errors")), is(notNullValue()));

Below is how to run our Acceptance Tests from Maven:
mvn test -DargLine=""

To run the Acceptance Tests from an IDE you will need to configure a VM argument like

You can of course change the driver so you use different browsers. As the available drivers[1] support javascript from a front end perspective you can come up with very complete test assertions that later can be scheduled so you keep checking the website functionality in a regular basis.

I have to say though that to test Front End development you better wait for the preference of your UI Engineer. Java is most likely not his preferred language.

While I here provide guidance to ATDD from a Java perspective I still think that given the fact that ATDD is closer to front end development or User experience (UX) it should be better to study other possibilities in the case you have the luxury of having a UX engineer on board.