訪問hdfs裡的檔案
阿新 • • 發佈:2018-11-12
準備工作:
給hdfs裡上傳一份用於測試的檔案
[[email protected] ~]# cat hello.txt
hello 1
hello 2
hello 3
hello 4
[[email protected] ~]# hadoop fs -put ./hello.txt /
[[email protected] ~]# hadoop fs -ls /
Found 1 items
-rw-r--r-- 2 root supergroup 32 2018-11-12 22:42 /hello.txt
java依賴的庫:
1.common
hadoop-2.7.3\share\hadoop\common\hadoop-common-2.7.3.jar
2.common依賴的jar
hadoop-2.7.3\share\hadoop\common\lib下的所有
3.hdf
hadoop-2.7.3\share\hadoop\hdfs\hadoop-hdfs-2.7.3.jar
程式碼:
利用JDK的URL類
import org.apache.hadoop.io.IOUtils; import org.apache.hadoop.fs.FsUrlStreamHandlerFactory;import java.io.InputStream; import java.net.URL; public class TestHDFS { public static void main(String[] args) throws Exception{ // URL url = new URL("http://www.baidu.com"); //URL這個類是Java的,他預設只認識HTTP協議,這裡需要設定一下,讓他認識HDFS協議 URL.setURLStreamHandlerFactory(new FsUrlStreamHandlerFactory());//這裡的地址和埠,相當與hdfs裡的根目錄, 然後在拼上要訪問的檔案在hdfs裡的路徑 URL url = new URL("hdfs://192.168.0.104:9000/hello.txt"); InputStream in = url.openStream(); IOUtils.copyBytes(in, System.out, 4096, true); } }