Wednesday, June 13, 2012

Hadoop, POB and Error: JAVA_HOME is not set

As I recently posted the importance of knowing Plain Old Bash (POB) shell scripting is huge, way more than what many IT Ops and Devs think.

Hadoop, which has been making history lately in pretty much any article you read about Big Data is heavily based on simple concepts and at the same time tightly tied to Bash. Yes POB scripts are the ones responsible for part of the Hadoop implementation Magic (Besides Java of course).

So I see some people complaining about the below error:
Error: JAVA_HOME is not set
And then some people suggesting to hardcode the $JAVA_HOME variable in $HADOOP_INSTALL/conf/hadoop-env.sh besides exporting $JAVA_HOME as a "shell environment variable".

No need for duplication really.

In Debian/Ubuntu for example if you use pseudo distributed mode you just have to set your $JAVA_HOME in ~/.bashrc (instead of just in ~/.profile or ~/.bash_profile). The reason is that Hadoop uses not-login interactive shell (like ssh user@host /path/to/command) to run commands in remote nodes. Clearly a setting in ~/.profile or ~/.bash_profile will not be picked by an interactive not-login shell.

If you are running a fully distributed cluster then set JAVA_HOME in /etc/environment

2 comments:

Kai Wähner said...

You saved my day! Good and logical explanation also helps for understanding the problem. Thanks.

Best regards,
Kai

gaurav said...

Saved my Day..Thanks !!

Followers