Based on the documentation
http://hortonworks.com/kb/spark-1-1-0-technical-preview-hdp-2-1-5/
I got the following error,
[root@sandbox spark-1.1.0.2.1.5.0-695-bin-2.4.0.2.1.5.0-695]# ./bin/spark-submit –class org.apache.spark.examples.SparkPi –master yarn-cluster –num-executors 3 –driver-memory 512m –executor-memory 512m –executor-cores 1 lib/spark-examples*.jar 10
Spark assembly has been built with Hive, including Datanucleus jars on classpath
14/09/25 08:46:14 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform… using builtin-java classes where applicable
14/09/25 08:46:15 INFO impl.TimelineClientImpl: Timeline service address: http://sandbox.hortonworks.com:8188/ws/v1/timeline/
14/09/25 08:46:15 INFO client.RMProxy: Connecting to ResourceManager at sandbox.hortonworks.com/127.0.0.1:8050
14/09/25 08:46:16 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 0 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:17 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 1 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:18 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 2 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:19 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 3 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:20 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 4 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:21 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 5 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:22 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 6 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:23 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 7 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)
14/09/25 08:46:24 INFO ipc.Client: Retrying connect to server: sandbox.hortonworks.com/127.0.0.1:8050. Already tried 8 time(s); retry policy is RetryUpToMaximumCountWithFixedSleep(maxRetries=50, sleepTime=1000 MILLISECONDS)