hdfs 複製檔案 java.io.FileNotFoundException(檔名、目錄名或卷標語法不正確)
阿新 • • 發佈:2019-01-02
Exception in thread "main" java.io.FileNotFoundException: hdfs:\192.168.73.16:8020\user\9003547\text.txt (檔名、目錄名或卷標語法不正確。)
at java.io.FileInputStream.open(Native Method)
at java.io.FileInputStream.<init>(FileInputStream.java:146)
at org.chinaskin.hadoop.FileCopy.main(FileCopy.java:29)
在win上使用Myeclipse連結hadoop時出現如上錯誤,注意看列印的錯誤資訊,傳入的是 hdfs://192.xxxx,變成了hdfs:\192xxxxx,將傳入的兩個引數打印出來發現並無異常並且檔案在hdfs上是存在的。
程式碼如下:
import java.io.InputStream;
import java.io.OutputStream;
import java.io.BufferedInputStream;
import java.io.FileInputStream;
import java.net.URI;
import org.apache.hadoop.conf.Configuration;
import org.apache.hadoop.fs.FileSystem;
import org.apache.hadoop.fs.Path;
import org.apache.hadoop.io.IOUtils;
public class aa {
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
FileSystem fs = FileSystem.get(URI.create(args[1]), conf);
InputStream in = new BufferedInputStream(new FileInputStream(args[0]));
OutputStream out = fs.create(new Path(args[1]));
IOUtils.copyBytes(in, out, 4096, true);
}
}
public class aa {
static {// 靜態塊,設定hdfs協議
URL. setURLStreamHandlerFactory ( new FsUrlStreamHandlerFactory());
}
public static void main(String[] args) throws Exception {
Configuration conf = new Configuration();
URL url = new URL(args[0]);
InputStream in = url.openStream();//引入相應的包使用url讀取資料
FileSystem fs = FileSystem.get(URI.create(args[1]), conf);
OutputStream out = fs.create(new Path(args[1]));
IOUtils.copyBytes(in, out, 4096, true);
}
}
另一種解決方法,其實找不到檔名的錯誤是因為,在將一個檔案讀取轉換為位元組流的時候,沒有經過hadoop 的FileSystem,即
```java
InputStream in = new BufferedInputStream(new FileInputStream(args[0]));
```
修改為InputStream in = new BufferedInputStream(fs.open(new Path(args[0])));
讀取hdfs的檔案使用 fs.open(enw Path(args[0]))