Please check this thread for more info.
When I’m executing a distcp command to copy from local file system to hdfs, I’m getting the following error :
15/06/01 13:53:37 INFO tools.DistCp: DistCp job-id: job_1431689151537_0003
15/06/01 13:53:37 INFO mapreduce.Job: Running job: job_1431689151537_0003
15/06/01 13:53:44 INFO mapreduce.Job: Job job_1431689151537_0003 running in uber mode : false
15/06/01 13:53:44 INFO mapreduce.Job: map 0% reduce 0%
15/06/01 13:53:47 INFO mapreduce.Job: Task Id : attempt_1431689151537_0003_m_000000_1000, Status : FAILED
java.io.FileNotFoundException: File /opt/dev/sdb/hadoop/yarn/local/filecache does not exist
15/06/01 13:53:51 INFO mapreduce.Job: Task Id : attempt_1431689151537_0003_m_000000_1001, Status : FAILED
java.io.FileNotFoundException: File /opt/dev/sdd/hadoop/yarn/local/filecache does not exist
15/06/01 13:53:55 INFO mapreduce.Job: Task Id : attempt_1431689151537_0003_m_000000_1002, Status : FAILED
java.io.FileNotFoundException: File /opt/dev/sdh/hadoop/yarn/local/filecache does not exist
15/06/01 13:54:02 INFO mapreduce.Job: map 100% reduce 0%
15/06/01 13:54:02 INFO mapreduce.Job: Job job_1431689151537_0003 completed successfully
15/06/01 13:54:02 INFO mapreduce.Job: Counters: 34
File System Counters
FILE: Number of bytes read=41194
FILE: Number of bytes written=118592
FILE: Number of read operations=0
FILE: Number of large read operations=0
FILE: Number of write operations=0
HDFS: Number of bytes read=372
HDFS: Number of bytes written=41194
HDFS: Number of read operations=15
HDFS: Number of large read operations=0
HDFS: Number of write operations=4
Job Counters
Failed map tasks=3
Launched map tasks=4
Other local map tasks=4
Total time spent by all maps in occupied slots (ms)=10090
Total time spent by all reduces in occupied slots (ms)=0
Total time spent by all map tasks (ms)=10090
Total vcore-seconds taken by all map tasks=10090
Total megabyte-seconds taken by all map tasks=10332160
Map-Reduce Framework
Map input records=1
Map output records=0
Input split bytes=114
Spilled Records=0
Failed Shuffles=0
Merged Map outputs=0
GC time elapsed (ms)=31
CPU time spent (ms)=760
Physical memory (bytes) snapshot=223268864
Virtual memory (bytes) snapshot=2503770112
Total committed heap usage (bytes)=1171259392
File Input Format Counters
Bytes Read=258
File Output Format Counters
Bytes Written=0
org.apache.hadoop.tools.mapred.CopyMapper$Counter
BYTESCOPIED=41194
BYTESEXPECTED=41194
COPY=1