MapReduce服务 MRS-文件错误导致上传文件到HDFS失败:原因分析

时间:2023-11-01 16:19:38

原因分析

  1. 查看NameNode日志“/var/log/Bigdata/hdfs/nn/hadoop-omm-namenode-主机名.log”,发现该文件一直在被尝试写,直到最终失败。
    2015-07-13 10:05:07,847 | WARN  | org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@36fea922 | DIR* NameSystem.internalReleaseLease: Failed to release lease for file /hive/order/OS_ORDER._8.txt._COPYING_. Committed blocks are waiting to be minimally replicated. Try again later. | FSNamesystem.java:39362015-07-13 10:05:07,847 | ERROR | org.apache.hadoop.hdfs.server.namenode.LeaseManager$Monitor@36fea922 | Cannot release the path /hive/order/OS_ORDER._8.txt._COPYING_ in the lease [Lease.  Holder: DFSClient_NONMAPREDUCE_-1872896146_1, pendingcreates: 1] | LeaseManager.java:459org.apache.hadoop.hdfs.protocol.AlreadyBeingCreatedException: DIR* NameSystem.internalReleaseLease: Failed to release lease for file /hive/order/OS_ORDER._8.txt._COPYING_. Committed blocks are waiting to be minimally replicated. Try again later.at FSNamesystem.internalReleaseLease(FSNamesystem.java:3937)
  2. 根因分析:被上传的文件损坏,因此会上传失败。
  3. 验证办法:cp或者scp被拷贝的文件,也会失败,确认文件本身已损坏。
support.huaweicloud.com/trouble-mrs/mrs_03_0082.html