MapReduce服务 MRS-FileInputFormat split的时候出现数组越界:问题

时间:2023-11-01 16:25:29

问题

HDFS调用FileInputFormat的getSplit方法的时候,出现ArrayIndexOutOfBoundsException: 0,日志如下:

java.lang.ArrayIndexOutOfBoundsException: 0at org.apache.hadoop.mapred.FileInputFormat.identifyHosts(FileInputFormat.java:708)at org.apache.hadoop.mapred.FileInputFormat.getSplitHostsAndCachedHosts(FileInputFormat.java:675)at org.apache.hadoop.mapred.FileInputFormat.getSplits(FileInputFormat.java:359)at org.apache.spark.rdd.HadoopRDD.getPartitions(HadoopRDD.scala:210)at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:239)at org.apache.spark.rdd.RDD$$anonfun$partitions$2.apply(RDD.scala:237)at scala.Option.getOrElse(Option.scala:120)at org.apache.spark.rdd.RDD.partitions(RDD.scala:237)at org.apache.spark.rdd.MapPartitionsRDD.getPartitions(MapPartitionsRDD.scala:35)
support.huaweicloud.com/cmpntguide-mrs/mrs_01_1700.html