I would suggest you double-check permissions on hdfs:///tmp
Interestingly, I’m trying to get the same scenario running, but I packaged Hortonworks HDFS in separate Docker containers (namenodes and datanodes) and Spark 1.3.1 in the standalone mode into another Docker container. And I’m facing yet another problem: com.google.protobuf.InvalidProtocolBufferException: Protocol message end-group tag did not match expected tag. Have you seen this along your way?