Big Data Series: Copying files onto and off-of HDFS

I have a file called words.txt in my downloads folder

Open command line

run cd Downloads to change directory to the one in which your file is saved and run ls to ensure that the file you want is there.

We can copy our file to HDFS by typing ‘hadoop fs -copyFromLocal words.txt’, we can then check that the copy has completed successfully by typing ‘hadoop fs -ls’.

We can also choose to make a copy of a file in HDFS by using ‘hadoop fs -cp words.txt words2.txt’ and we can check that it’s copied successfully by running ‘hadoop fs -ls’

We can then copy our new words2.txt file back to our local system by using ‘hadoop fs -copyToLocal words2.txt’ and we can run ‘ls’ to check that the file has been copied

We can also remove files from Hadoop easily by running ‘hadoop fs -rm words2.txt’. We can run ‘hadoop fs -ls’ to check that it’s been deleted.

 

 

Kieran Keene

view all posts

Join me on this career development project as I set out to develop the skills required to progress up the technology career ladder! Check out http://netshock.co.uk/about/ to find out more.