Hadoop Commands(Part 2)
- Command used for conditional checking.
>>We can use multiple parameters with test command they are d,s,f,e,r,w,z.
Command : hadoop fs -test -d directoryname
>>To check the result for the above command we have to use echo $?
Output:
•-d → Check whether the path given by the user is a directory or not, return 0 if it is a directory.
• -f → Check whether the path given by the user is a file or not, return 0 if it is a file.
• -e → Check whether the path given by the user exists or not, return 0 if the path exists.
• -z → Checks whether the file size is 0 byte or not, return 0 if the file is of 0 bytes.
• -w →Return 0 if the path exists and write permission is granted.
• -r → Return 0 if the path exists and read permission is granted.
• -s → Check if the path is not empty, return 0 if a path is not empty.
• And the result will be in the form of 0 and 1.
0 → we will get output as 0 if the directory is present.
1 → we will get output as 1 if the directory is not present.
2. To Move file from hdfs to local system and vice versa.
Commands :
- hadoop fs -moveFromLocal filelocation (present in local system) location (of hadoop file system).
- hadoop fs -moveToLocal filelocation (present in hadoop system) location (of local file system).
Output:
3. To Merge the files of Hadoop file system to Local file system.
Command:
hadoop fs -getmerge (filename) file location (file name present in hadoop system) ~/localfile system location/ merged file (as per our choice)
Output:
4. To append files of the local system to one of the files present in Hadoop.
Command : hadoop fs -appendToFile ( files location of local file system ) (file of hdfs location)
Output:
5. To check the integrity of a file.
➢ Such as whether the file has been modified or not.
➢ Hashing concept is used in this command. (MD5 Algo is used — Message Digest)
➢ Output of this command will be in alpha-numeric.
Command : hadoop fs -checksum file location
Output :
6. To check the health status of the Hadoop System of a particular directory and the entire root directory.
fsck→ file system check.
Command : hdfs fsck — /
hdfs fsck -/dir1
7. To check the count of files and directories present in the given location.
Command: hadoop fs -count directoryname
Output:
From the above output of hadoop fs -count dir1 explanation:
1 → number of directories.
5 → number of files.
8507124 → size occupied by those files
8. To delete a Directory.
Command : hadoop fs -rmr directoryname
Output:
9.To delete a File
Command: hadoop fs -rm file location
Output:
10. To know Statistics of a file
Command: hadoop fs -stat %b file location
we can use multiple parameters:
%r→to know the replication factor of a file
%b→to know the number of bytes of the file
%g→to know the group name
%u→to know the username of a file
%y→to know when the file is modified
Output:
11. To change the group of a file.
Command: hadoop fs -chgrp groupname(own choice) filelocation
Output:
12. To change the permissions of a file or directory.
Command: hadoop fs -chmod 777(any rule you want to give) filelocation or directory
>> You can give 777,754,755…etc., any rule at your convenience.
Output:
