Hello, I am having an issue with exporting to S3, wondering if you could give any advice….
I get an error saying:
2015-04-27 05:39:49,547 INFO [IPC Server handler 0 on 40333] org.apache.hadoop.mapred.TaskAttemptListenerImpl: Diagnostics report from attempt_1429544880663_0004_m_000000_0: Error: java.io.IOException: Could not get the output FileSystem with root=s3n://<ACCESS_KEY_ID>:<SECRET_ACCESS_KEY>@<BUCKET_NAME>/2HBASE-SNAP_X
at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.setup(ExportSnapshot.java:149)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:142)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:784)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:163)
at java.security.AccessController.doPrivileged(Native Method)
at javax.security.auth.Subject.doAs(Subject.java:415)
at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1628)
at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:158)
Caused by: java.io.IOException: No FileSystem for scheme: s3n
at org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2584)
at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2591)
at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:91)
at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2630)
at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2612)
at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:370)
at org.apache.hadoop.hbase.snapshot.ExportSnapshot$ExportMapper.setup(ExportSnapshot.java:147)
... 8 more
My MapReduce isn’t my strongest and I have a feeling it could be to do with not specifying an outpath for the mappers?
The code I use to export the snapshot is:
hbase org.apache.hadoop.hbase.snapshot.ExportSnapshot -snapshot "SNAP_X" -copy-to s3n://<ACCESS_KEY_ID>:<SECRET_ACCESS_KEY>@<BUCKET_NAME>/2HBASE-SNAP_X -mappers 3