I have set up HDP cluster using Ambari 1.7. I am able to process 50GB of data (2750 tasks) on cluster.
But the problem come when i run another script to proces 50 GB the total tasks comes to 4300 . Job fails with Heap size error.
↧
Memory configuration for 1NN (16GB RAM ) and 4 DN (8GB RAM)
↧