Wednesday, January 25, 2012

Portable PGP: Encryption with Bouncy Castle

I came across the PPGP project Today and I thought about sharing it not only because it is a nice tool that runs in any OS (Java) which allows to encrypt/decrypt files and text using PGP, sign the result, generate keys but also because being Open Source Java Project the whole source code is available so you could use proven code that actually works to perform PGP operations in your own project.

Below is simple code almost entirely extracted from this project that just encrypts a file with a PGP public key.

package com.nestorurquiza.utils;

import java.io.FileInputStream;
import java.io.FileOutputStream;
import java.io.InputStream;
import java.io.OutputStream;
import java.security.SecureRandom;
import java.security.Security;
import java.util.Date;
import java.util.Iterator;

import org.bouncycastle.bcpg.ArmoredOutputStream;
import org.bouncycastle.jce.provider.BouncyCastleProvider;
import org.bouncycastle.openpgp.PGPCompressedData;
import org.bouncycastle.openpgp.PGPCompressedDataGenerator;
import org.bouncycastle.openpgp.PGPEncryptedData;
import org.bouncycastle.openpgp.PGPEncryptedDataGenerator;
import org.bouncycastle.openpgp.PGPLiteralData;
import org.bouncycastle.openpgp.PGPLiteralDataGenerator;
import org.bouncycastle.openpgp.PGPPublicKey;
import org.bouncycastle.openpgp.PGPPublicKeyRing;
import org.bouncycastle.openpgp.PGPPublicKeyRingCollection;
import org.bouncycastle.openpgp.PGPUtil;

/**
 * 
 * Code in part extracted from http://sourceforge.net/projects/ppgp/ 
 * A nice GUI to generate your own keys, encrypt and sign documents using PGP
 * 
 * @author nestor
 *
 */
public class PgpUtils {
    private final static int BUFFER_SIZE = 1 << 16;

    private static PGPPublicKey readPublicKeyFromCollection2(InputStream in) throws Exception {
        in = PGPUtil.getDecoderStream(in);
        PGPPublicKeyRing pkRing = null;
        PGPPublicKeyRingCollection pkCol = new PGPPublicKeyRingCollection(in);
        Iterator it = pkCol.getKeyRings();
        while (it.hasNext()) {
             pkRing = (PGPPublicKeyRing) it.next();
             Iterator pkIt = pkRing.getPublicKeys();
             while (pkIt.hasNext()) {
                     PGPPublicKey key = (PGPPublicKey) pkIt.next();
                     if (key.isEncryptionKey())
                             return key;
             }
        }
        return null;
    }
    

    /**
     * Encrypts a file using a public key
     * 
     * @param decryptedFilePath
     * @param encryptedFilePath
     * @param encKeyPath
     * @param armor
     * @param withIntegrityCheck
     * @throws Exception
     */
    public static void encryptFile(String decryptedFilePath,
            String encryptedFilePath,
            String encKeyPath,
            boolean armor,
            boolean withIntegrityCheck)            
            throws Exception{

        OutputStream out = new FileOutputStream(encryptedFilePath);
        FileInputStream pubKey = new FileInputStream(encKeyPath);
        PGPPublicKey encKey = readPublicKeyFromCollection2(pubKey);
        Security.addProvider(new BouncyCastleProvider());
        
        if (armor) 
            out = new ArmoredOutputStream(out);
            
        // Init encrypted data generator
        PGPEncryptedDataGenerator encryptedDataGenerator =
                new PGPEncryptedDataGenerator(PGPEncryptedData.CAST5, withIntegrityCheck, new SecureRandom(),"BC");
        
        encryptedDataGenerator.addMethod(encKey);
        
        
        OutputStream encryptedOut = encryptedDataGenerator.open(out, new byte[BUFFER_SIZE]);

        // Init compression  
        PGPCompressedDataGenerator compressedDataGenerator = new PGPCompressedDataGenerator(PGPCompressedData.ZIP);
        OutputStream compressedOut = compressedDataGenerator.open(encryptedOut);  

        PGPLiteralDataGenerator literalDataGenerator = new PGPLiteralDataGenerator();
        OutputStream literalOut = literalDataGenerator.open(compressedOut, PGPLiteralData.BINARY, decryptedFilePath, new Date(), new byte[BUFFER_SIZE]);
        FileInputStream inputFileStream = new FileInputStream(decryptedFilePath);
        byte[] buf = new byte[BUFFER_SIZE];  
        int len;
        while((len = inputFileStream.read(buf))>0){
            literalOut.write(buf,0,len);
        }
         
        literalOut.close();
        literalDataGenerator.close();
        
        compressedOut.close();
        compressedDataGenerator.close();
        encryptedOut.close();
        encryptedDataGenerator.close();
        inputFileStream.close();
        out.close();
        
    }
}

The Unit test showing how to use it:
package com.nestorurquiza.utils;

import java.io.IOException;

import org.junit.Test;
import org.xml.sax.SAXException;

public class PgpUtilsTest {
    @Ignore
    @Test    
    public void encryptTest() throws IOException, Exception, SAXException {
        String publicKeyFilePath = "/Users/nestor/Downloads/somePublicKey.pgp";
        String decryptedFilePath = "/Users/nestor/Downloads/someFile.csv";
        String encryptedFilePath = "/Users/nestor/Downloads/someFile.csv.enc";
        
        PgpUtils.encryptFile(decryptedFilePath, encryptedFilePath, publicKeyFilePath, false, true);
        
    }
}

Tuesday, January 24, 2012

Commons VFS: SFTP from Java the simple way

I have always used jsch library to SCP (or what is commonly known as SFTP) files in remote servers from Java.

I was tempted to use my old code when I decide to look around for something cleaner just to come across this great post about sftp using Apache VFS. I made few changes to be able to sftp any input stream as a file to a remote SFTP site which I am sharing below.

Dependencies:

                org.apache.commons
                commons-vfs2
                2.0
            
            
                com.jcraft
                jsch
                0.1.45
            

The upload:

package com.nestorurquiza.utils;

package com.nestorurquiza.utils;

import java.io.InputStream;
import java.io.OutputStream;

import org.apache.commons.io.IOUtils;
import org.apache.commons.vfs2.FileObject;
import org.apache.commons.vfs2.FileSystemException;
import org.apache.commons.vfs2.FileSystemOptions;
import org.apache.commons.vfs2.Selectors;
import org.apache.commons.vfs2.impl.StandardFileSystemManager;
import org.apache.commons.vfs2.provider.sftp.SftpFileSystemConfigBuilder;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
 * All credits for http://www.memorylack.com/2011/06/apache-commons-vfs-for-sftp.html
 * 
 * nestoru - 2012/01/24: Small changes:
 *   1. Using logging instead of System.out.println()
 *   2. The initial directory is not the user home directory 
 *   3. No additional slashes when adding remoteFilePath.
 *   4. Using InputStream instead of path to a local file
 * 
 * If you need extra functionality check out the url above
 * 
 * 
 *
 */
public class SftpUtils {
    private final static Logger log = LoggerFactory.getLogger(SftpUtils.class);
    
    public static void upload(String hostName, String username,
            String password, InputStream localInputStream, String localInputStreamName, String remoteFilePath) {
        
        StandardFileSystemManager manager = new StandardFileSystemManager();
        FileObject localFile;
        FileObject remoteFile;

        try {
            manager.init();

            FileSystemOptions fileSystemOptions = createDefaultOptions();
            
            // Create local file object
            //FileObject localFile = manager.resolveFile(f.getAbsolutePath());
            localFile = manager.resolveFile("ram://path/needed/" + localInputStreamName);
            localFile.createFile();
            OutputStream localOutputStream = localFile.getContent().getOutputStream();
            IOUtils.copy(localInputStream, localOutputStream);
            localOutputStream.flush();
            
            // Create remote file object
            remoteFile = manager.resolveFile(
                    createConnectionString(hostName, username, password,
                            remoteFilePath), fileSystemOptions);

            // Copy local file to sftp server
            remoteFile.copyFrom(localFile, Selectors.SELECT_SELF);

            log.debug("File upload success");
        } catch (Exception e) {
            throw new RuntimeException(e);
        } finally {
            remoteFile != null ? remoteFile.close();
            localFile != null ? localFile.close();
            manager.close();
        }
    }
    
    public static String createConnectionString(String hostName,
            String username, String password, String remoteFilePath) {
        // result: "sftp://user:123456@domainname.com/resume.pdf
        return "sftp://" + username + ":" + password + "@" + hostName
                + remoteFilePath;
    }

    public static FileSystemOptions createDefaultOptions() throws FileSystemException {
        // Create SFTP options
        FileSystemOptions opts = new FileSystemOptions();
        
        // SSH Key checking
        SftpFileSystemConfigBuilder.getInstance().setStrictHostKeyChecking(
                opts, "no");
        
        // Root directory set to user home
        SftpFileSystemConfigBuilder.getInstance().setUserDirIsRoot(opts, false);
        
        // Timeout is count by Milliseconds
        SftpFileSystemConfigBuilder.getInstance().setTimeout(opts, 10000);
        
        return opts;
    }

}


Then a quick unit test that I comment out Just because I am not interested really in regression test but rather just testing if the method works as expected (I do not like main() methods inside the classes)
package com.nestorurquiza.utils;

import java.io.File;
import java.io.FileInputStream;
import java.io.IOException;

import org.junit.Test;
import org.xml.sax.SAXException;


public class SftpUtilsTest {
    //@Ignore
    @Test    
    public void uploadTest() throws IOException, Exception, SAXException {
        String hostName = "bhubint.nestorurquiza.com";
        String userame = "user";
        String password = "pass";
        String localFilePath = "/tmp/test.txt";
        String remoteFilePath = "/home/report/report/test.txt";
        //SftpUtils.upload(hostName, userame, password, localFilePath, remoteFilePath);
        File f = new File(localFilePath);
        if (!f.exists())
            throw new RuntimeException("Error. Local file not found");
        FileInputStream fileInputStream = new FileInputStream(f);
        SftpUtils.upload(hostName, userame, password, fileInputStream, "test.txt", remoteFilePath);
    }
}

maven downloads the pom.xml but not the jar file

Today I spent sometime troubleshooting one particular library I uploaded to Artifactory and which was not downloaded by Maven.

Maven was downloading just the pom file instead of downloading the jar file.

I had to reupload the jar and while doing it edit the pom file to include the pacjaging=jar
jar

Tuesday, January 17, 2012

Complex development environments with VirtualBox

Having a local environment independent of any other existing environments has a big advantage: Development is not stopped when other environments are down and it even stops the "need" to hit production just because it is the only environment currently available. But perhaps the biggest advantage of all is to be able to work literally anywhere even if you have no internet connection at all.

Virtualization can help with this and here is how to host your whole environment locally in your machine using VirtualBox.

In an Enterprise you will find Solaris, Windows, OSX, Unix and what not. I personally use a Mac Book Pro with VirtualBox. In VirtualBox I can run as many VMs as I need to recreate the surrounded environment.

Ideally we should have an environment following the next features:
  1. The guest should have connectivity to internet taking advantage of your network configuration in the host
  2. The guest should communicate to the host even if no internet connection is available
  3. The guest should automatically work independent what network you are using, from work, from home or even on the road with no local network at all

Virtual box will allow these features if we just add a first network interface as NAT (for internet) and the second as host-only (for connectivity with the box)

We will show this process using a virtualized instance of an Advent Geneva Server (A Solaris Box with some proprietary Accounting Software). It is useful to have this server locally to be able to mess with RSL reports for example without affecting other members of the dev or operations team.

In the host find the vboxnet0 interface to be sure you use an IP in the same network later on in the guest:
$ ifconfig -a
...
vboxnet0: flags=8843 mtu 1500
 ether 0a:00:27:00:00:00 
 inet 192.168.56.1 netmask 0xffffff00 broadcast 192.168.56.255
... 

In the Solaris guest ...

Configure first interface (NAT) to use DHCP
$ touch /etc/dhcp.e1000g0
$ rm /etc/hostname.e1000g0
$ touch /etc/hostname.e1000g0
$ ifconfig e1000g0 plumb
$ ifconfig e1000g0 up
$ ifconfig e1000g0 dhcp start

And second interface (host-only) to use a static IP
$ vi /etc/hosts
...
192.168.56.2 genevadev loghost
...
$ rm /etc/dhcp.e1000g1
$ vi /etc/hostname.e1000g0
192.168.56.2
$ ifconfig e1000g1 plumb
$ ifconfig e1000g1 genevadev up
...

Be sure to define DNS available from any network to connect to the internet (only to be used in the case you do need internet in the guest) If you really need to access local resources you might want to add local DNS as well.
$ vi /etc/resolv.conf
nameserver 72.45.32.34
nameserver 72.45.32.37

Make sure you see both interfaces (the NAT and host-only). They should show up as e1000g0 and e1000g1. The first with an internal 10.0.xxx.xxx IP and the second with the static IP you picked (for this example 192.168.56.2)
$ ifconfig -a

Clearly the static IP must be in the same subnetwork (192.168.56.xxx).

You should be able to connect to the internet from the guest and at the same time ssh into it using its 192.168.56.2 IP. Do not forget to add in your host the new created server resolution:
$ vi /etc/hosts
genevadev 192.168.56.2

The question about sharing this environment or building it from scratch is one interesting topic. An even more interesting is left as homework and it deals with devops. In reality regardless the path you follow you should automate your actions and there is where things like http://vagrantup.com/ and puppet/chef can help you further.

Thursday, January 12, 2012

Remove duplicate classes the agile way: Maven Duplicate Finder Plugin

Without discussing the alternatives I think a Java project cannot live without checking for dependency conflicts. The conflicts can be buried in external jar files so the check must be performed by fully qualified name on absolutely all classes included in the project.

Of course such a check should stop the build until a resolution.

The Maven Duplicate Finder Plugin does exactly that.

Just clone the repo, build the maven project and upload the plugin jar file to your Maven repository. Then configure your application to use the plugin:
...
<build>
  <plugins>
   <plugin>
              <groupId>com.ning.maven.plugins</groupId>
              <artifactId>maven-duplicate-finder-plugin</artifactId>
              <configuration>
                <failBuildInCaseOfConflict>false</failBuildInCaseOfConflict>
              </configuration>
              <executions>
                <execution>
                  <phase>verify</phase>
                  <goals>
                    <goal>check</goal>
                  </goals>
                </execution>
              </executions>
            </plugin>
...

If you run it with the failBuildInCaseOfConflict flag turned off as above you can get the messages as WARNING without stopping the project from building.
[INFO] [duplicate-finder:check {execution: default}]
[INFO] Checking compile classpath
[WARNING] Found duplicate classes in [commons-beanutils:commons-beanutils:1.8.3,commons-collections:commons-collections:3.2.1] :
[WARNING]   org.apache.commons.collections.ArrayStack
[WARNING]   org.apache.commons.collections.Buffer
[WARNING]   org.apache.commons.collections.BufferUnderflowException
[WARNING]   org.apache.commons.collections.FastHashMap
[WARNING] Found duplicate classes in [net.objectlab.kit:datecalc-common:1.2.0,net.objectlab.kit:datecalc-jdk:1.2.0] :
[WARNING]   net.objectlab.kit.datecalc.common.AbstractDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.AbstractIMMDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.AbstractKitCalculatorsFactory
[WARNING]   net.objectlab.kit.datecalc.common.DateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.DefaultHolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.ExcelDateUtil
[WARNING]   net.objectlab.kit.datecalc.common.HolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.HolidayHandler
[WARNING]   net.objectlab.kit.datecalc.common.HolidayHandlerType
[WARNING]   net.objectlab.kit.datecalc.common.IMMDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.IMMPeriod
[WARNING]   net.objectlab.kit.datecalc.common.ImmutableHolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.KitCalculatorsFactory
[WARNING]   net.objectlab.kit.datecalc.common.PeriodCountBasis
[WARNING]   net.objectlab.kit.datecalc.common.PeriodCountCalculator
[WARNING]   net.objectlab.kit.datecalc.common.StandardTenor
[WARNING]   net.objectlab.kit.datecalc.common.Tenor
[WARNING]   net.objectlab.kit.datecalc.common.TenorCode
[WARNING]   net.objectlab.kit.datecalc.common.Utils
[WARNING]   net.objectlab.kit.datecalc.common.WorkingWeek
[WARNING] Found duplicate resources in [net.sf.jasperreports:jasperreports:4.1.3,net.sf.jasperreports:jasperreports-fonts:4.1.3] :
[WARNING]   jasperreports_extension.properties
[WARNING] Found duplicate resources in [org.bouncycastle:bcmail-jdk14:1.38,org.bouncycastle:bcprov-jdk14:1.38,org.bouncycastle:bctsp-jdk14:1.38] :
[WARNING]   META-INF/BCKEY.DSA
[WARNING]   META-INF/BCKEY.SF
[INFO] Checking runtime classpath
[WARNING] Found duplicate classes in [commons-beanutils:commons-beanutils:1.8.3,commons-collections:commons-collections:3.2.1] :
[WARNING]   org.apache.commons.collections.ArrayStack
[WARNING]   org.apache.commons.collections.Buffer
[WARNING]   org.apache.commons.collections.BufferUnderflowException
[WARNING]   org.apache.commons.collections.FastHashMap
[WARNING] Found duplicate classes in [net.objectlab.kit:datecalc-common:1.2.0,net.objectlab.kit:datecalc-jdk:1.2.0] :
[WARNING]   net.objectlab.kit.datecalc.common.AbstractDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.AbstractIMMDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.AbstractKitCalculatorsFactory
[WARNING]   net.objectlab.kit.datecalc.common.DateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.DefaultHolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.ExcelDateUtil
[WARNING]   net.objectlab.kit.datecalc.common.HolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.HolidayHandler
[WARNING]   net.objectlab.kit.datecalc.common.HolidayHandlerType
[WARNING]   net.objectlab.kit.datecalc.common.IMMDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.IMMPeriod
[WARNING]   net.objectlab.kit.datecalc.common.ImmutableHolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.KitCalculatorsFactory
[WARNING]   net.objectlab.kit.datecalc.common.PeriodCountBasis
[WARNING]   net.objectlab.kit.datecalc.common.PeriodCountCalculator
[WARNING]   net.objectlab.kit.datecalc.common.StandardTenor
[WARNING]   net.objectlab.kit.datecalc.common.Tenor
[WARNING]   net.objectlab.kit.datecalc.common.TenorCode
[WARNING]   net.objectlab.kit.datecalc.common.Utils
[WARNING]   net.objectlab.kit.datecalc.common.WorkingWeek
[WARNING] Found duplicate resources in [net.sf.jasperreports:jasperreports:4.1.3,net.sf.jasperreports:jasperreports-fonts:4.1.3] :
[WARNING]   jasperreports_extension.properties
[WARNING] Found duplicate resources in [org.bouncycastle:bcmail-jdk14:1.38,org.bouncycastle:bcprov-jdk14:1.38,org.bouncycastle:bctsp-jdk14:1.38] :
[WARNING]   META-INF/BCKEY.DSA
[WARNING]   META-INF/BCKEY.SF
[INFO] Checking test classpath
[WARNING] Found duplicate classes in [commons-beanutils:commons-beanutils:1.8.3,commons-collections:commons-collections:3.2.1] :
[WARNING]   org.apache.commons.collections.ArrayStack
[WARNING]   org.apache.commons.collections.Buffer
[WARNING]   org.apache.commons.collections.BufferUnderflowException
[WARNING]   org.apache.commons.collections.FastHashMap
[WARNING] Found duplicate classes in [net.objectlab.kit:datecalc-common:1.2.0,net.objectlab.kit:datecalc-jdk:1.2.0] :
[WARNING]   net.objectlab.kit.datecalc.common.AbstractDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.AbstractIMMDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.AbstractKitCalculatorsFactory
[WARNING]   net.objectlab.kit.datecalc.common.DateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.DefaultHolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.ExcelDateUtil
[WARNING]   net.objectlab.kit.datecalc.common.HolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.HolidayHandler
[WARNING]   net.objectlab.kit.datecalc.common.HolidayHandlerType
[WARNING]   net.objectlab.kit.datecalc.common.IMMDateCalculator
[WARNING]   net.objectlab.kit.datecalc.common.IMMPeriod
[WARNING]   net.objectlab.kit.datecalc.common.ImmutableHolidayCalendar
[WARNING]   net.objectlab.kit.datecalc.common.KitCalculatorsFactory
[WARNING]   net.objectlab.kit.datecalc.common.PeriodCountBasis
[WARNING]   net.objectlab.kit.datecalc.common.PeriodCountCalculator
[WARNING]   net.objectlab.kit.datecalc.common.StandardTenor
[WARNING]   net.objectlab.kit.datecalc.common.Tenor
[WARNING]   net.objectlab.kit.datecalc.common.TenorCode
[WARNING]   net.objectlab.kit.datecalc.common.Utils
[WARNING]   net.objectlab.kit.datecalc.common.WorkingWeek
[WARNING] Found duplicate resources in [net.sf.jasperreports:jasperreports:4.1.3,net.sf.jasperreports:jasperreports-fonts:4.1.3] :
[WARNING]   jasperreports_extension.properties
[WARNING] Found duplicate resources in [org.bouncycastle:bcmail-jdk14:1.38,org.bouncycastle:bcprov-jdk14:1.38,org.bouncycastle:bctsp-jdk14:1.38] :
[WARNING]   META-INF/BCKEY.DSA
[WARNING]   META-INF/BCKEY.SF

The above is the result of a project that I cleaned up so actually the errors you are seeing are the result of "accepted" conflicts. Since we are aware of them then we proceed (at our own risk) to exclude them from being checked as exceptions while setting failBuildInCaseOfConflict to true. That way no developer will be able to include a conflicting dependency without resolving it first.
...
<build>
  <plugins>
    <plugin>
              <groupId>com.ning.maven.plugins</groupId>
              <artifactId>maven-duplicate-finder-plugin</artifactId>
              <configuration>
                <failBuildInCaseOfConflict>true</failBuildInCaseOfConflict>
                <ignoredResources>
                  <ignoredResource>META-INF.*</ignoredResource>
                </ignoredResources>
                <exceptions>
                  <exception>
                    <conflictingDependencies>
                      <dependency>
                        <groupId>commons-beanutils</groupId>
                        <artifactId>commons-beanutils</artifactId>
                      </dependency>
                      <dependency>
                        <groupId>commons-collections</groupId>
                        <artifactId>commons-collections</artifactId>
                      </dependency>
                    </conflictingDependencies>
                    <packages>
                      <package>org.apache.commons.collections</package>
                    </packages>
                  </exception>
                  <exception>
                    <conflictingDependencies>
                      <dependency>
                        <groupId>net.objectlab.kit</groupId>
                        <artifactId>datecalc-common</artifactId>
                      </dependency>
                      <dependency>
                        <groupId>net.objectlab.kit</groupId>
                        <artifactId>datecalc-jdk</artifactId>
                      </dependency>
                    </conflictingDependencies>
                    <packages>
                      <package>net.objectlab.kit.datecalc.common</package>
                    </packages>
                  </exception>
                  <exception>
                    <conflictingDependencies>
                      <dependency>
                        <groupId>net.sf.jasperreports</groupId>
                        <artifactId>jasperreports</artifactId>
                      </dependency>
                      <dependency>
                        <groupId>net.sf.jasperreports</groupId>
                        <artifactId>jasperreports-fonts</artifactId>
                      </dependency>
                    </conflictingDependencies>
                    <resources>
                      <resource>jasperreports_extension.properties</resource>
                    </resources>
                  </exception>
                </exceptions>
              </configuration>
              <executions>
                <execution>
                  <phase>verify</phase>
                  <goals>
                    <goal>check</goal>
                  </goals>
                </execution>
              </executions>
     </plugin>
...

Note how I have globally ignored any resource inside META-INF and I have ignored the properties file explicitly from the artifacts where they were found.

Also I have not used the classes node which would allow me to specify the individual classes that I approve to ignore. This can be bad but I feel lazy today ;-)

Finally I did not specify the version number for the conflicting dependencies. This can be bad as well.

Forgot Password with LDAP, CAS and Spring: Credentials expired

I have already documented the steps I followed to integrate existing Spring applications that happen to be using LDAP with CAS including how to support "Forgot Password" functionality.

Some CAS client applications are currently using certain complicated logic to handle Terms and Conditions, Forgot Password and other features through custom filters and they were failing with a 401 ERROR "User credentials have expired". Basically the user hits the Forgot Password link in CAS which shows a CAS Client page where the email is given and after submission the temporary password is mailed. Any attempt to move to any page from that moment on results in a 401.

The problem here is that the default implementation when using Spring CAS client is to test the forcePasswordChange LDAP attribute and it currently throws an Exception rather than easily allowing to hook into the current implementation let us say with a property like "IGNORE_CREDENTIALS_EXPIRATION". As the project is already handling password expiration it does not need that extra check from Spring CAS Client so after some research I had to hack into Spring code.

I found that a possible solution is to use a CustomCasAuthenticationProvider to override CasAuthenticationProvider#authenticateNow() however I had to copy the whole code from the latter because of scope problems. Basically the original class is not designed to be inherited (at least for this feature). Below is the only method I changed in the class:
    private CasAuthenticationToken authenticateNow(final Authentication authentication) throws AuthenticationException {
        try {
            final Assertion assertion = this.ticketValidator.validate(authentication.getCredentials().toString(), serviceProperties.getService());
            final UserDetails userDetails = loadUserByAssertion(assertion);
            try {
                userDetailsChecker.check(userDetails);
            } catch (CredentialsExpiredException e) {
                if (log.isDebugEnabled()) {
                    log.debug("Credentials expired but we handle that case outside of the CAS logic.");
                }
            }
            return new CasAuthenticationToken(this.key, userDetails, authentication.getCredentials(), userDetails.getAuthorities(), userDetails, assertion);
        } catch (final TicketValidationException e) {
            throw new BadCredentialsException(e.getMessage(), e);
        }
    }

Monday, January 09, 2012

SSO with LDAP Spring and CAS

This is a step by step documentation on how to setup a CAS Server and use Single Sign On (SSO) plus Single Sign Off from Spring client applications using LDAP to store authentication and authorization information. CAS support can be enabled or disabled commenting sections of the code.

Our Use Case for this exercise is to have a centralized server that handles user login and logout operations allowing the user to sign in just once and enjoy multiple applications thanks to that centralization.

We then achieve Federation because thanks to SSO via CAS we obtain Authentication while we pull the attributes from LDAP to achieve Authorization. At the end the user is trusted after the first login and the attributes can be obtained from any application.

Before we review Server and Client configuration be sure you understand CAS needs SSL so in order to correctly follow this post you will need to have the CAS Server and two Spring client applications serving on https. SSL needs an IP per domain and if you are running OSX by default you have only one loopback IP (127.0.0.1). This means you will need to add other two:
sudo ifconfig lo0 alias 127.0.0.2 up
sudo ifconfig lo0 alias 127.0.0.3 up

Then set in /etc/hosts a domain per loopback IP:
$ vi /etc/hosts
127.0.0.2   bhubdev.nestorurquiza.com
127.0.0.2   portaldev.nestorurquiza.com
127.0.0.3   casdev.nestorurquiza.com

You should be able to get the certificate from each URL and assert they are correct, for example:
$ echo | openssl s_client -connect bhubdev.nestorurquiza.com:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/Downloads/bhubdev.nestorurquiza.com.cer
$ echo | openssl s_client -connect portaldev.nestorurquiza.com:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/Downloads/portaldev.nestorurquiza.com.cer
$ echo | openssl s_client -connect casdev.nestorurquiza.com:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/Downloads/casldev.nestorurquiza.com.cer
$ openssl x509 -text -in ~/Downloads/bhubdev.nestorurquiza.com.cer
$ openssl x509 -text -in ~/Downloads/portaldev.nestorurquiza.com.cer
$ openssl x509 -text -in ~/Downloads/casdev.nestorurquiza.com.cer

Once you are sure the certificates coming from the different URLs are correct (they belong to the specific domain) then you should add them to your keystore:
$ sudo keytool -delete -alias bhubdev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts
$ sudo keytool -delete -alias portaldev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts
$ sudo keytool -delete -alias casdev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts
$ sudo keytool -import -alias bhubdev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts -file ~/Downloads/bhubdev.nestorurquiza.com.cer
$ sudo keytool -import -alias portaldev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts -file ~/Downloads/portaldev.nestorurquiza.com.cer
$ sudo keytool -import -alias casdev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts -file ~/Downloads/casdev.nestorurquiza.com.cer

Of course after you do all the above tomcat must be restarted and if you use Apache for virtual hosting like I explain below you owe to restart it as well.

CAS Server installation and configuration

  1. Download CAS Server from http://www.jasig.org/cas and get cas.war deployed in your server. At this point http://localhost:8080/cas should prompt for credentials. I commonly use apache in front of tomcat so below are the steps using virtual hosts as I have documented before:
    $ mkdir /Users/nestor/Downloads/apache-tomcat-7.0.22/cas
    $ mkdir /Users/nestor/Downloads/apache-tomcat-7.0.22/conf/Catalina/casdev.nestorurquiza.com
    $ cp /Users/nestor/Downloads/cas-server-3.4.11/modules/cas-server-webapp-3.4.11.war /Users/nestor/Downloads/apache-tomcat-7.0.22/cas/ROOT.war
    
  2. Prepare CAS to use LDAP authentication. This is about copying two files to the WEB-INF/lib directory and editing a configuration file in that same directory:
    $ cp /Users/nestor/Downloads/cas-server-3.4.11/modules/cas-server-support-ldap-3.4.11.jar /Users/nestor/Downloads/apache-tomcat-7.0.22/cas/ROOT/WEB-INF/lib 
    $ cp /Users/nestor/Downloads/spring-ldap-1.3.0.RELEASE/dist/spring-ldap-1.3.0.RELEASE-all.jar /Users/nestor/Downloads/apache-tomcat-7.0.22/cas/ROOT/WEB-INF/lib
    $ vi /Users/nestor/Downloads/apache-tomcat-7.0.22/cas/ROOT/WEB-INF/deployerConfigContext.xml
    …
     
     ...
      
       
        
        
        
        
        
        
          
          
          
        
       
      
    
  3. Get spring-security source code





    ldaps://ldapint.nestorurquiza.com:10636













  4. While at this point you should be able to login from the CAS landing page using your LDAP credentials CAS needs to run from an SSL URL in order to be used as Single Sign On (SSO). I use Apache and mod-jk with Tomcat so my SSL configuration is achieved on the Apache side. Here are the steps to generate the self signed certificate and configure Apache virtual host in OSX to serve the CAS service:
    $ cd ~/Downloads/
    $ sudo mkdir /private/etc/apache2/certs/
    $ export DOMAIN=casdev.nestorurquiza.com
    $ openssl genrsa -des3 -out ${DOMAIN}.key 1024
    $ openssl req -new -key ${DOMAIN}.key -out  ${DOMAIN}.csr
    $ openssl x509 -req -days 365 -in ${DOMAIN}.csr -signkey ${DOMAIN}.key -out ${DOMAIN}.crt
    $ openssl rsa -in ${DOMAIN}.key -out ${DOMAIN}.key
    $ sudo cp ${DOMAIN}.key /private/etc/apache2/certs/
    $ sudo cp ${DOMAIN}.crt /private/etc/apache2/certs/
    $ sudo vi /private/etc/apache2/extra/httpd-vhosts-ssl.conf
    Listen 443
    AddType application/x-x509-ca-cert .crt
    AddType application/x-pkcs7-crl    .crl
    SSLPassPhraseDialog  builtin
    SSLSessionCache        "shmcb:/private/var/run/ssl_scache(512000)"
    SSLSessionCacheTimeout  300
    SSLMutex  "file:/private/var/run/ssl_mutex"
    
    
    DocumentRoot "/usr/docs/casdev.nestorurquiza.com"
    ServerName casdev.nestorurquiza.com:443
    ServerAdmin nurquiza@nestorurquiza.com
    ErrorLog "/private/var/log/apache2/casdev.nestorurquiza.com.log"
    TransferLog "/private/var/log/apache2/casdev.nestorurquiza.com.log"
    SSLEngine on
    SSLCipherSuite ALL:!ADH:!EXPORT56:RC4+RSA:+HIGH:+MEDIUM:+LOW:+SSLv2:+EXP:+eNULL
    SSLCertificateFile "/private/etc/apache2/certs/casdev.nestorurquiza.com.crt"
    SSLCertificateKeyFile "/private/etc/apache2/certs/casdev.nestorurquiza.com.key"
    
        Options +FollowSymLinks
        AllowOverride All
        Order deny,allow
        Allow from all
    
    JkMount "/*" nestorurquiza-app1
    
    $ sudo vi /etc/apache2/httpd.conf
    ...
    LoadModule ssl_module libexec/apache2/mod_ssl.so
    ...
    Include /private/etc/apache2/extra/httpd-vhosts-ssl.conf
    …
    $ sudo apachectl configtest
    $ sudo apachectl graceful
    
  5. Logout and login from SSL. Note that you do not get any alerts stating SSO won't work:
    https://casdev.nestorurquiza.com/logout
    https://casdev.nestorurquiza.com
    
  6. Change the look and feel for the CAS login page editing the file /WEB-INF/view/jsp/default/ui/casLogoutView.jsp and dependencies like the logo from /css/cas.css #header background url and #header h1#app-name background color
  7. Further customize the login page for example as I needed to provide a forgot password functionality I came up with the below implementation in casLogoutView.jsp:
      
      
    <a href="<c:out value="${schema}://${domain}/forgotPassword"/>">Forgot Password</a>
  8. Import the client website certificate in the CAS server for example:
    $ echo | openssl s_client -connect bhubdev.nestorurquiza.com:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/Downloads/bhubdev.nestorurquiza.com.cer
    $ sudo keytool -import -alias bhubdev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts -file ~/Downloads/bhubdev.nestorurquiza.com.cer
    



Spring CAS Client

  1. Import the CAS certificate in the client
    $ echo | openssl s_client -connect casdev.nestorurquiza.com:443 2>/dev/null | sed -ne '/-BEGIN CERTIFICATE-/,/-END CERTIFICATE-/p' > ~/Downloads/casdev.nestorurquiza.com.cer
    $ sudo keytool -import -alias casdev.nestorurquiza.com -keystore  /Library/Java/Home/lib/security/cacerts -file ~/Downloads/casdev.nestorurquiza.com.cer
    
  2. Declare some properties in the classpath of the client application:
    ...
    #
    # CAS
    #
    cas.loginURL=https://casdev.nestorurquiza.com/login
    cas.homeURL=https://casdev.nestorurquiza.com
    cas.serviceURL=https://bhubdev.nestorurquiza.com/j_spring_cas_security_check
    cas.logoutURL=https://casdev.nestorurquiza.com/logout?service=https%3A%2F%2Fbhubdev.nestorurquiza.com%2Flogout
    …
    
  3. Then configure the client to perform the Single Sign On and Single Sign Off through CAS. Here is the Spring security configuration for such a use case:
    ...
        
        
        
        
        
            
            
            
            
     
            
            
            
            
                       
        
        
            
            
        
        
        
          
        
        
        
          
          
        
        
        
          
        
        
        
          
          
        
        
        
            
            
            
              
                
                
            
            
        
    
        
    
  4. A simple JSP View for the client application Logout Controller
    <%@ include file="/WEB-INF/jsp/includes.jsp" %>
    <%@ include file="/WEB-INF/jsp/header.jsp" %>
    
    

    <%@ include file="/WEB-INF/jsp/footer.jsp" %>

  5. The LogoutController
    package com.nestorurquiza.web;
    
    import javax.servlet.http.HttpServletRequest;
    import javax.servlet.http.HttpServletResponse;
    
    import org.springframework.security.core.context.SecurityContextHolder;
    import org.springframework.stereotype.Controller;
    import org.springframework.web.bind.annotation.RequestMapping;
    import org.springframework.web.servlet.ModelAndView;
    
    @Controller
    public class LogoutController extends RootController {
    
        @RequestMapping("/logout")
        public ModelAndView welcomeHandler(HttpServletRequest request,
                HttpServletResponse response) {
            SecurityContextHolder.getContext().setAuthentication(null);
            
            ...
            
            return getModelAndView(ctx, "logout", null);
        }
    }
    
  6. The logout link in JSP:
    
                            
                                
                            
                           
                                
    


  7. At least one dependency is needed in the client application:
    
                org.springframework.security
                spring-security-cas-client
                ${spring-security.version}
                compile
            
    


  8. At this point hitting any client application URL (for example https://bhubdev.nestorurquiza.com/client/home) will redirect the user to the CAS login URL passing the service parameter which contains the encoded servciceURL (https://casdev.nestorurquiza.com/login?service=https%3A%2F%2Fbhubdev.nestorurquiza.com%2Fj_spring_cas_security_check). When the user provides valid credentials CAS will redirect her back to the j_spring_cas_security_check client resource and Spring will take care to pull from the session the originally requested URL. When the user clicks on the logout URL (https://casdev.nestorurquiza.com/logout?service=https%3A%2F%2Fbhubdev.nestorurquiza.com%2Flogout) the CAS server will invalidate the security token and the user will return to a URL where a custom Logout Controller invalidates the Spring Security Context. At this point any attempt to access a protected resource will result in a redirect to the CAS login page as explained.


  9. Probably you will need in your custom code to do extra logic with the saved request (the original requested URL). In Spring 3.0.3 at least you use the below snippet to gain access to it:
    SavedRequest savedRequest = new DefaultSavedRequest(request, new PortResolverImpl());
    if( savedRequest != null ) {
      String redirectUrl = savedRequest.getRedirectUrl();
    


  10. Clearly you can proceed with the final step from the Server and the steps from the Client to configure extra applications (CAS Services). When you logout from one application or directly from CAS interface you will be logged out from all applications that are integrated with CAS (Single Sign Out or Single Sign Off functionality)


Other than a little issue I reported here the whole integration went smooth. CAS is well supported in Spring and both Single Sign On and Single Sign Off will work out of the box.

By default CAS is shipped with a 4 hours expiration ticket expiration policy. This can be configured as explained in the documentation https://wiki.jasig.org/display/CASUM/Ticket+Expiration+Policy

Thursday, January 05, 2012

Access a properties file from JSP/JSTL EL

Sometimes you need to show information in a JSP from a properties file. Typical scenario is configuration data.

With spring you could do:


Then from a Listener or even from a Controller you could Autowire and inject the variable as a context or request attribute respectively:
...
@Autowired
public Properties applicationProperties;
...
request.setAttribute("applicationProperties", applicationProperties);

Suppose app.properties has this content:
...
bhub.environment=LOCAL
...

Then from JSP/JSTL you can use it as:

Subversive Authentication error svn OPTIONS 403 Forbidden

Suddenly Eclipse started to complaint with an error like the below when I tried to update certain local svn copies:
Authentication error.
svn: OPTIONS of '/repos/reporting': 403 Forbidden (http://subversion.nestorurquiza.com)

While someone could be tempted to grant more permisions on the SVN server the reality is that from command line I did not get the same experience. Everything was working just fine from there.

To correct this I used the Subversive Update site from Eclipse Help/Install Software Menu. After an Eclipse restart I had to pick the latest 1.6 svn connector (not native) for it to work:
http://download.eclipse.org/technology/subversive/0.7/update-site/ - [required] Subversive plug-in

Followers