HDFS的常用命令操作
1、基本語法
2、常用命令操作
#######################
1、基本語法
bin/hadoop fs 具體命令
或
bin/hdfs dfs 具體命令
（dfs是fs的實現類）

二者區別

hadoop fs：可以作用於除hdfs外的其他檔案系統，作用範圍更廣。通用的檔案系統命令，針對任何系統，比如本地檔案、HDFS檔案、HFTP檔案、S3檔案系統等。
（曾經還有hadoop dfs：專門針對hdfs分散式檔案系統，已經不推薦使用）

hdfs dfs：專門針對hdfs分散式檔案系統，相比於上面的命令更為推薦，並且當使用hadoop dfs時內部會被轉為hdfs dfs命令。
2、常用命令操作
》啟動hadoop叢集
sbin/start-dfs.sh
sbin/start-yarn.sh
（1）-help：可數出命令所需的引數
hadoop@hadoop101:/opt/module/hadoop-3.1.3/bin$ hadoop fs -help rm
-rm [-f] [-r|-R] [-skipTrash] [-safely] <src> ... :
Delete all files that match the specified file pattern. Equivalent to the Unix
command "rm <src>"

-f If the file does not exist, do not display a diagnostic message or
modify the exit status to reflect an error.
-[rR] Recursively deletes directories.
-skipTrash option bypasses trash, if enabled, and immediately deletes <src>.
-safely option requires safety confirmation, if enabled, requires
confirmation before deleting large directory with more than
<hadoop.shell.delete.limit.num.files> files. Delay is expected when
walking over large directory recursively to count the number of
files to be deleted before the confirmation.

（2）-ls：顯示目錄資訊
hadoop@hadoop101:/opt/module/hadoop-3.1.3/bin$ hadoop fs -ls /
Found 1 items
drwxr-xr-x - hadoop supergroup 0 2020-02-29 00:11 /home
（3）-mkdir：在HDFS上建立目錄
hadoop@hadoop101:/opt/module/hadoop-3.1.3/bin$ hadoop fs -mkdir -p /home/hadoop/op_test
hadoop@hadoop101:/opt/module/hadoop-3.1.3/bin$ hadoop fs -ls /home/hadoop
Found 2 items
drwxr-xr-x - hadoop supergroup 0 2020-02-29 00:14 /home/hadoop/input
drwxr-xr-x - hadoop supergroup 0 2020-03-02 06:42 /home/hadoop/op_test
（4）-moveFromLocal：從本地剪下貼上到HDFS
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ touch op_test.txt
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ echo "hello,hdfs" >> op_test.txt
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -moveFromLocal ./op_test.txt /home/hadoop/op_test
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ ls
test.input

（5）-appendToFile：追加一個檔案到已經存在的檔案末尾
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ touch op_append.txt
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ echo "append-hdfs" >> op_append.txt
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop -appendToFile op_append.txt /home/hadoop/op_test/op_test.txt
（6）-cat：顯示檔案內容
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -cat /home/hadoop/op_test/op_test.txt
hello,hdfs
append-hdfs
（7）-chgrp、-chmod、-chown：Linux檔案系統中的用法一樣，修改檔案所述許可權
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -chmod 666 /home/hadoop/op_test/op_test.txt
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -chown hadoop:hadoop /home/hadoop/op_test/op_test.txt

（8）-copyFromLocal：從本地檔案系統中拷貝檔案到HDFS路徑去
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ ls
op_append.txt test.input
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -copyFromLocal test.input /home/hadoop/op_test

（9）-copyToLocal：從HDFS拷貝到本地
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -copyToLocal /home/hadoop/op_test/op_test.txt ./
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ ls
op_append.txt op_test.txt test.input
（10）-cp：從HDFS的一個路徑拷貝到HDFS的另一個路徑
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -cp /home/hadoop/op_append.txt /home/hadoop/op_test

（11）-mv：在HDFS目錄中移動檔案
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -mv /home/hadoop/op_test/op_append.txt /home/hadoop
（12）-get：等同於copyToLocal，從HDFS下載檔案到本地
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -get /home/hadoop/op_append.txt ./
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ ls
op_append.txt op_test.txt test.input
（13）-getmerge：從HDFS的目錄下，將檔案合併起來後下載
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -getmerge /home/hadoop/op_test/* ./common.txt

hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ ls
common.txt op_append.txt op_test.txt test.input

hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ cat common.txt
append-hdfs
hello,hadoop!
（14）-put：等同於copyFromLocal，從HDFS目錄下拷貝檔案到本地
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -put ./common.txt /home/hadoop/op_test/
2020-03-02 07:15:57,049 INFO sasl.SaslDataTransferClient: SASL encryption trust check: localHostTrusted = false, remoteHostTrusted = false
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -ls /home/hadoop/op_test
Found 3 items
-rw-r--r-- 3 hadoop supergroup 26 2020-03-02 07:15 /home/hadoop/op_test/common.txt
-rw-r--r-- 3 hadoop supergroup 12 2020-03-02 07:06 /home/hadoop/op_test/op_append.txt
-rw-r--r-- 3 hadoop supergroup 14 2020-03-02 06:56 /home/hadoop/op_test/test.input
（15）-tail：顯示一個檔案的末尾
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -tail
home/hadoop/op_test/common.txt
append-hdfs
hello,hadoop!
（16）-rm：刪除檔案或資料夾
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -rm /home/hadoop/*.txt
Deleted /home/hadoop/op_append.txt
Deleted /home/hadoop/op_test.txt
（17）-rmdir：刪除空目錄
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -mkdir /test
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -rmdir /test
（18）-du：統計資料夾的大小資訊
-h：顯示帶MB及以上的可讀形式
-s：得到資料夾下檔案大小的總和
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -du /home/hadoop/op_test
26 78 /home/hadoop/op_test/common.txt
12 36 /home/hadoop/op_test/op_append.txt
14 42 /home/hadoop/op_test/test.input
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -du -h /home/hadoop/op_test
26 78 /home/hadoop/op_test/common.txt
12 36 /home/hadoop/op_test/op_append.txt
14 42 /home/hadoop/op_test/test.input
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -du -s /home/hadoop/op_test
52 156 /home/hadoop/op_test
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -du -s -h /home/hadoop/op_test

（19）-setrep：設定HDFS中檔案的副本數量
hadoop@hadoop101:/opt/module/hadoop-3.1.3/testinput$ hadoop fs -setrep 2 /home/hadoop/op_test/common.txt
Replication 2 set: /home/hadoop/op_test/common.txt
1
2

注：實際可存在的副本數受DataNode的數量限制，不可大於DataNode的數量。若設定的副本數大於DataNode的數量，只在NameNode中記錄本次設定的數量，而實際的副本數量將自動被控制為與DataNode相同數量。