Jack,
You don’t need the full hive-site.xml in Spark. Please see http://hortonworks.com/hadoop-tutorial/using-apache-spark-hdp/
You only need the following section in the hive site under spark.
Running Hive 0.13.1 UDF
Before running Hive examples run the following steps:
Create hive-site in Spark conf
Create the file SPARK_HOME/conf/hive-site.xml.
Edit the file to contain only the following statements: