HADOOP BASIC COMMANDS

The installation of Hadoop (single node cluster) , starting and stopping of Hadoop cluster etc. was mentioned in my previous post.

Some of the basic Hadoop Commands are listed below,

List Commands:

hadoop dfs -ls </source path>

To list the files present in the root directory:

hadoop fs -ls /

To list the files present in the home directory:

hadoop fs -ls ./

List the contents of the directory:

hadoop fs -ls

hadoop fs -ls </path>

eg:-

hadoop fs -ls /user/hduser

Put Command:

hadoop dfs –put </source path> </destination path>

To upload a file:

hadoop fs -put ./localfile.txt /home/Documents/remotefile.txt

Get Command:

hadoop dfs –get </source path> </destination path>

To retrieve a file:

hadoop fs -get /home/Documents/remotefile.txt ./local/file/path/file.txt

View the contents of a file:

hadoop dfs –cat </path[filename]>

Make Directory Command:

hadoop fs -mkdir -p [directory name]

eg:-

hadoop fs -mkdir -p dir1

hadoop fs -mkdir <paths>

eg:-

hadoop fs -mkdir /user/hduser/dir2

hadoop dfs -mkdir -p  [directory name]

eg:-

hadoop dfs -mkdir -p dir3

 Remove an empty directory:

hadoop fs -rmdir [directory name]

hadoop dfs -rm -r [directory name]

Duplicating a Complete File inside the HDFS:

hadoop dfs –copyFromLocal </source path> </destination path>

Duplicating a File from HDFS to the Local File System

hadoop dfs –copyToLocal </source path> </destination path>

Removing the File:

hadoop dfs –rm </path[filename]>

Run a DFS Filesystem to Check Utility:

hadoop fsck </file path>

Run a Cluster Balancing Utility:

hadoop balancer

Check Directory Space in HDFS:

hadoop fs -du -s -h </file path>

eg:-

hadoop fs -du -s -h dir2

To find a directory/File in Hadoop:

hdfs dfs -ls -R / | grep [search_term]

To move the files:

hadoop fs -mv </source> </destination>

eg:-

hadoop fs -mv /user/hduser/dir2 /user/hduser/dir1

List all the Hadoop File System Shell Commands:

hadoop fs [options]

Help:

hadoop fs –help

Empty the Trash:

hadoop fs –expunge

You can refer the Hadoop commands Guide here.

Advertisements

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s