ICP 1 - PallaviArikatla/Big-Data-Programming GitHub Wiki

Name : Pallavi Arikatla

Topic: Installation/Cloudera / Hue

This ICP is about moving files from local file system to HDFS and on some basic commands:

Question:

Load it in hadoop hdfs

Create a directory in hdfs using following command:

hadoop fs -mkdir icp1

Now load the local input file into HDFS with the below command:

hdfs dfs -put /home/cloudera/Downloads/shakespeare.txt icp1/

Append file word_list to the first file using below command:

hdfs dfs -appendToFile /home/cloudera/Downloads/word_list.txt icp1/shakespeare.txt

Then visualized the data using hue

View the first and last lines using head and tail commands as shown below:

hadoop fs icp1/shakespeare.txt | head hadoop fs -tail icp1/shakespeare.txt

Create a new text file and load it into hdfs and try to append all three datasets using below commands

hadoop fs -getmerge icp1/* icp1/finalfile

hadoop fs -put icp1/finalfile icp1/