Quantcast
Channel: Hortonworks » All Replies
Viewing all articles
Browse latest Browse all 3435

Reply To: hadoop 2.6 installation/setup, hdfs ok, yarn/mapred usaly fail but sometime ok

$
0
0

ok, i get it, my cluster work fine on all datatnode :)

i make install oozie 4.1.0 for solve my issue, but i must recompile with the correct version hdoop

http://hadooptutorial.info/apache-oozie-installation-on-ubuntu-14-04/#comment-1470

http://mockus.in/forum/viewtopic.php?t=48

but i must install maven 3.2.1 instead of maven 2.2.1 because i get fail message oozie-hadoop is duplicate in reactor

mvn clean package assembly:single -P hadoop-2 -DskipTests

i fixe all my container resource, because i installed more memory on my master server up from 8GB to 24GB ram.

python /home/hduser/yarn-util.py -c 4 -m 12 -d 3 -k False
Using cores=4 memory=12GB disks=3 hbase=False
Profile: cores=4 memory=11264MB reserved=1GB usableMem=11GB disks=3
Num Container=6
Container Ram=1536MB
Used Ram=9GB
Unused Ram=1GB
yarn.scheduler.minimum-allocation-mb=1536
yarn.scheduler.maximum-allocation-mb=9216
yarn.nodemanager.resource.memory-mb=9216
mapreduce.map.memory.mb=1536
mapreduce.map.java.opts=-Xmx1228m
mapreduce.reduce.memory.mb=3072
mapreduce.reduce.java.opts=-Xmx2457m
yarn.app.mapreduce.am.resource.mb=3072
yarn.app.mapreduce.am.command-opts=-Xmx2457m
mapreduce.task.io.sort.mb=614

Now i am very happy and i can start to play with some toys of hadoop ecosystem (hbase,hive,mahoot,pig,…)

Issue close for me
KR
JP


Viewing all articles
Browse latest Browse all 3435

Trending Articles



<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>