1. 程式人生 > >hdfs檔案讀寫bug2

hdfs檔案讀寫bug2

  多DFSClient同時寫同一個檔案,可能會導致以下異常。其中一個會獲取許可權寫,另外的應該會報異常。(但是獲取許可權的順序不太確定,也麼有深究)

 IOExceptionLease mismatch on  xxxxx owned by DFSClient_NONMAPREDUCE_230060699_8but is accessed by DFSClient_NONMAPREDUCE_1017678927_8

         atorg.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2949)

         atorg.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)

         atorg.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)

         atorg.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)

         atorg.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)

         atorg.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

         atorg.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)

         atorg.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)

         atorg.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)

         atorg.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)

         atjava.security.AccessController.doPrivileged(Native Method)

         atjavax.security.auth.Subject.doAs(Subject.java:415)

         atorg.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)

         atorg.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)

 多DFSClient同時寫同一個檔案,報出異常之後,自以為是的在異常捕獲中加入了將遠端檔案(上傳的相同檔案)刪除的程式碼。結果又出了另一個bug。一個在上傳,另一個把這個檔案刪除了。。。。

  IOException---No lease on   /XXXX: File does not exist. HolderDFSClient_NONMAPREDUCE_-223137850_8 does not have any open files.

atorg.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:2934)

atorg.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:2754)

atorg.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:2662)

atorg.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:584)

atorg.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:440)

atorg.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)

atorg.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:585)

atorg.apache.hadoop.ipc.RPC$Server.call(RPC.java:928)

atorg.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2013)

atorg.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2009)

atjava.security.AccessController.doPrivileged(Native Method)

atjavax.security.auth.Subject.doAs(Subject.java:415)

atorg.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1548)

atorg.apache.hadoop.ipc.Server$Handler.run(Server.java:2007)