Wednesday, February 24, 2010

Serving external content from your own domain with Apache mod_proxy and mod_rewrite

Ok so you want to serve images from your Service provider but you have an SSL site or simply you just want for the sake of uniformity to serve those images from your own domain.

- Oh boy!,  someone says, we need to build a python script that uses curl and download the images, that will take a while ... and bandwidth.
- OK we can cache, said a tech guy.
- Project is then delayed? freaked out the PM
- We need to negotiate ASAP, said the BA while running to his desk to write an email.
- ...

How many solutions a problem might have? Commonly more than one. What we need here has a name and actually a Pattern Name. We need a proxy, and apache has a proxy module, and that can be integrated with the rewrite module and so we can happily serve any image from our own domain even though the resource is located in an external server.

So in the apache conf file for the particular Virtual Host we include a rewrite rule:
<VirtualHost *>
     ServerName images.mydomain.com
     #In production systems comment the below two lines
     RewriteLogLevel 3
     RewriteLog "logs/images.mydomain.com.rewrite.log"

     ...
     <Directory "/my/virtual/host/docroot/directory">
        ...
        RewriteRule (.*) http://externaldomain.com/%1 [P,L]
        ...
    </Directory>
    ...
</VirtualHost>
We hit http://images.mydomain.com/img.png. Bad luck, it does not work and from the log file we see:
File does not exist: proxy:http://externaldomain.com/img.png

Let us take a look at Server libraries
httpd -l
 Voila, mod-proxy support is not enabled. So after we recompile apache with support for mod-proxy:
./configure --prefix=/usr/local/apache --enable-so --enable-rewrite=shared --enable-ssl=shared --enable-proxy --enable-proxy-connect  --enable-proxy-http
make
make install


We do get what we expected (mod_proxy.c) and our image http://images.mydomain.com/img.png is showing up even though it does not exist in our domain but in the external server.

Wednesday, February 17, 2010

Hot code replace failed - Scheme change not implemented

This issue has been hitting my team for a while, some people like me were more lucky and some were simply not getting "hot code replace" working in the server while changing it from Eclipse.

And I was truly lucky using hot deployments for Tomcat and JBoss so far up to Today when after installing Eclipse in my personal MAC I found out myself on the same boat:

Hot code replace failed - Scheme change not implemented
 Well, without getting into too much detail this happens when the Server is running classes that were not compiled with the same compiler Eclipse is using.

Of course you need to run your application server with the below flags (>=JDK 5)to be able to connect to a debug port and hot deploy:
JAVA_OPTS="$JAVA_OPTS -Xdebug -agentlib:jdwp=transport=dt_socket,address=8000,server=y,suspend=n"

But wait a second, why it did not happen to me before? It turned out to be the way I usually worked:

1. I use Maven to build from command prompt and the Maven Eclipse Plugin to build from Eclipse.
2. I have Eclipse building automatically and the compiled code goes into the Maven target directory.
3. I deploy into Tomcat / JBoss using custom maven profiles and I do run them from command prompt.
 
The above configuration would fail to "hot code replace" so what do I do different on my workplace MAC?

In my workplace I use a local Ubuntu VM running on my MAC and there I have a script that will *not* clean the target when building. I was trying maven using the clean option in my personal MAC and so my compiler was different from the one in use by Eclipse.

So if we are sure that we never clean from Maven but only from Eclipse the steps above will work amazingly good. Now I know why my crew was suffering and for sure they will get more Agile from now on.

wget or curl

This is not a comparison between the two tools but rather how to get them working the same way when requesting for example Web Services.

So here we are trying to test our super cool and light  JSON Service from the command prompt and curl comes to our rescue:
$ curl http://my-app.com/default/login?ert=json
Wget will need a couple of switches to provide the same output (for more info: wget --help):
$ wget -q -O - http://my-app.com/default/login?ert=json
Let's say you want to download a file. With wget you go this way:
wget -o http://google.com/images/srpr/logo4w.png
With curl you go with the below. Note the -L switch to make sure curl follows redirections:
curl -L -o logo4w.png http://google.com/images/srpr/logo4w.png

Saturday, February 13, 2010

No module named yum

I usually do not document installation problems even though I run into them every so often. This one though has taken so much time to get resolved that I decided to document.

So here is the operation system:
# cat /etc/redhat-release
Red Hat Enterprise Linux Server release 5.3 (Tikanga)
 And here is the problem:
#yum
There was a problem importing one of the Python modules
required to run yum. The error leading to this problem was:

   No module named yum

Please install a package which provides this module, or
verify that the module is installed correctly.

It's possible that the above module doesn't match the
current version of Python, which is:
... 
 Of course yum is clearly corrupted and needs to be reinstalled, so as root we will download the corresponding rpm and install it. Note that use the magnificent CentOS rpms.:
mkdir /tmp/yum
cd /tmp/yum
http://mirror.centos.org/centos/5/os/i386/CentOS/yum-3.2.22-20.el5.centos.noarch.rpm
rpm -Uvh yum-3.2.22-20.el5.centos.noarch.rpm

Right there I got some missing dependencies so then I pulled them as well and since I downloaded everything into /tmp/yum I can build them all at once. Note that expat library is needed by python-elementtree however I did not find the rpm in CentOS repo:

wget http://mirror.centos.org/centos/5/os/i386/CentOS/yum-fastestmirror-1.1.16-13.el5.centos.noarch.rpm
wget http://mirror.centos.org/centos/5/os/x86_64/CentOS/python-elementtree-1.2.6-5.x86_64.rpm
wget ftp://ftp.muug.mb.ca/mirror/centos/5.4/updates/x86_64/RPMS/expat-1.95.8-8.3.el5_4.2.x86_64.rpm
rpm -Uvh *

Now finally yum is back:
# yum --version|head -1
3.2.22

But you need to install the repos. While you can do this manually and use different repos (Use one that uses GPG keys ) the easiest way is to just install an rpm to get the rpmforge repo:
rpm -Uhv http://apt.sw.be/redhat/el5/en/x86_64/rpmforge/RPMS/rpmforge-release-0.3.6-1.el5.rf.x86_64.rpm

Followers