Reply To: Bad substitution error with spark on HDP 2.2

I had the same problem. It seems that “hdp.version” is not correctly passed to YARN.

Using the Spark version provided by Hortonworks solves the problem (use the exact link they provide in their “Spark1.2 on HDP 2.2″ instruction). But it is a modified version of Spark 1.2 Tehcnology Preview, whereas the last Spark version is 1.3.

Here is a dirty workaround which work with Spark 1.3:

Copy your entire /etc/hadoop/conf somewhere else. Update YARN_CONF_DIR. Edit mapred-site.xml. Replace all instances of ${hdp.version} with the actual version number (2.2.0.0-2041). It should work.

However it would be nice if someone from Hortonworks could provide us with a clean solution.