Mindmajix

Configuration of Hadoop

Hadoop is supported by GNU/Linux platform and its flavors. Configurations are specified by resources. A resource contains a set of name/value pairs as XML data. Each resource is named by either a String or by a Path. If named by a String, then the classpath is examined for a file with that name. If named by a Path, then the local filesystem is examined directly, without referring to the classpath.

Setting environment variables for Hadoop:-

The next step is to set up the PATH environment variables. You can set them as following:

Login as root user

Screenshot_1821   Cmd: cd/usr/lib/Hadoop-0.20/conf

Screenshot_1821   Cmd: ls

Screenshot_1821   Cmd:Vi hadoop-env.sh

 Screenshot_1821  Copy the java path as

Export JAVA-HOME=/usr/java/jdhal.6

Export HADOOP-HEAPSIZE=2000

Screenshot_1821   Save

Screenshot_1821   Login as object user

Screenshot_1821   To check the hadoop version

     Cmd: hadoop version

To start the hadoop services:-

Login as object user.

Screenshot_1821   Cmd: sudo/etc/init. d/hadoop-0.20-namenode start

Screenshot_1821   Repeat the command for all the daemon such as data node, secondary name node, job tracker, task.

Adding the host name in hadoop configuration file:-

Login as root user.

Screenshot_1821   Cmd: cd/usr/lib/hadoop-0.20/conf.

Screenshot_1821   Cmd:vi cove-site.xml

Screenshot_1821   Add your host name instead of local host in the file.

Save

Formatting the Name Node:-

Screenshot_1821   Cd/usr/lib/hadoop/bin/

Screenshot_1821   Cmd:ls

Screenshot_1821   Before formatting the Name Node, we have to assign users to the daemons in HDFS

Screenshot_1821   Cmd: export HADOOP NAME NODE-USER=hdfs

Cmd: export Hadoop-Secondary Name Node-user=hdfs

Cmd: export Hadoop DATA NODE hdfs JDB TRACKER hdfs=map red

Log in as object user

Screenshot_1821   Cmd: cd/usr/lib/.hadoop/bin/

Screenshot_1821   Cmd :ls

Screenshot_1821   Cmd: hadoop Name Node-format

Screenshot_1821   To format the name node

Screenshot_1821   Start all the services for hdfs

Cmd: /start-all.sh

Screenshot_1821   Cmd: cd/

Screenshot_1821   Cmd: hadoop fs-ls/

Screenshot_1821   Now the hadoop configuration is finished.

Screenshot_1821   Cmd: jps[To check all the daemons]

Installation of Library files:-

Log in as root user

Screenshot_1821   Cmd: yum install hadoop-0.20*

 Screenshot_1821  Now, all the files and libraries which are related to hadoop [i.e. remains supported for those, which are not installed previously]

 

 


 

0 Responses on Configuration of Hadoop"

Leave a Message

Your email address will not be published. Required fields are marked *

Copy Rights Reserved © Mindmajix.com All rights reserved. Disclaimer.
Course Adviser

Fill your details, course adviser will reach you.