I am new to hdp. Current I am doing poc on HDP 2.2.
When I run distcp to copy file from s3 to hdfs,
hadoop distcp s3n://my-bucket/tmp /tmp
I have following error.
Error: java.io.Exception, no filesystem for scheme: s3n
I add following config to core-site.xml
<property>
<name>fs.s3n.impl</name>
<value>org.apache.hadoop.fs.s3native.NativeS3FileSystem</value>
<description>The FileSystem for s3n: (Native S3) uris.</description>
</property>
I got new error
Error: java.lang.RuntimeException: java.lang.ClassNotFoundException: Class org.apache.hadoop.fs.s3native.NativeS3FileSystem not found
I believe this class is shipped with hadoop 2.6
https://hadoop.apache.org/docs/current/api/org/apache/hadoop/fs/s3native/NativeS3FileSystem.html
Anyone have the same issue?
How to make this distcp successful?