Ram,
There are two ways to upgrade. You can upgrade HDP to HDP 2.3.2 with Ambari 2.1.2 and that will automatically upgrade Spark to 1.4.1
The second way is to manually only upgrade Spark to 1.4.1
What is the OS you are on? Can you ensure the repo URL you use in 1.3.2 (step 3.b) is from section 1.3.3 and for your OS?