Data Node Start Failure with Java Permission denied error

Hi Comrades,

I am facing an error in starting a Data Node from Ambari WEB interface, and I need your help.
As always the error message is very comlex and long, but first thing first. Even advices on partial issue will be appreciated.

Thank you so much in advance.

———————————————————–

Below are details:

### HDP 2.1 on Centos 6 / x86_64 GNU/Linux

### 2 Nodes Vagrant VMs:
centos1.corelli.local (Master, Name Node)
centos2.corelli.local (Slave, Data Node)

### Ambari Server is configured in Node#1
Passwordless SSH was configured from Node#1 to Node#1 itself and to Node#2.

### Symptom: All the installation steps succeeded but the last installation step – starting Data Node fails with the below log:

Fail: Execution of ‘ulimit -c unlimited; export HADOOP_LIBEXEC_DIR=/usr/lib/hadoop/libexec && /usr/lib/hadoop/sbin/hadoop-daemon.sh –config /etc/hadoop/conf start datanode’ returned 1. starting datanode, logging to /var/log/hadoop/hdfs/hadoop-hdfs-datanode-centos2.corelli.local.out
+ exec /usr/jdk64/jdk1.7.0_45/bin/java -Dproc_datanode -Xmx1024m -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/hdfs -Dhadoop.log.file=hadoop.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,console -Djava.library.path=:/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -Dhadoop.log.dir=/var/log/hadoop/hdfs -Dhadoop.log.file=hadoop-hdfs-datanode-centos2.corelli.local.log -Dhadoop.home.dir=/usr/lib/hadoop -Dhadoop.id.str=hdfs -Dhadoop.root.logger=INFO,RFA -Djava.library.path=:/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native:/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native/Linux-amd64-64:/usr/lib/hadoop/lib/native -Dhadoop.policy.file=hadoop-policy.xml -Djava.net.preferIPv4Stack=true -server -Xmx1024m -Dhadoop.security.logger=ERROR,DRFAS -Dhadoop.security.logger=INFO,RFAS org.apache.hadoop.hdfs.server.datanode.DataNode
/usr/lib/hadoop-hdfs/bin/hdfs: line 207: /usr/jdk64/jdk1.7.0_45/bin/java: Permission denied
/usr/lib/hadoop-hdfs/bin/hdfs: line 207: exec: /usr/jdk64/jdk1.7.0_45/bin/java: cannot execute: Permission denied

At the end of the log message it complaints Java Permission. So Here are Java settings in Node#2. I can manually run JAVA binary in the below path.

### Java setting in Node#2:

root@centos2~$ echo $JAVA_HOME
/usr/jdk64/jdk1.7.0_45/bin/java

root@centos2~$ ll /usr/jdk64/jdk1.7.0_45/bin/java
-rwxr-xr-x 1 uucp 143 7718 Oct 8 2013 /usr/jdk64/jdk1.7.0_45/bin/java

root@centos2~$ cat .bashrc
export PATH=/usr/jdk64/jdk1.7.0_45/bin/:$PATH
export JAVA_HOME=/usr/jdk64/jdk1.7.0_45/bin/java

root@centos2~$ java -version
java version “1.7.0_45″
Java(TM) SE Runtime Environment (build 1.7.0_45-b18)
Java HotSpot(TM) 64-Bit Server VM (build 24.45-b08, mixed mod