Quantcast
Channel: Hortonworks » All Replies
Viewing all articles
Browse latest Browse all 3435

Reply To: HDFS Rebalancing

$
0
0

It doesn’t rebalance itself, but HDFS includes a utility you can run that will do rebalancing in the background:
* Balancer User Guide – https://issues.apache.org/jira/secure/attachment/12370966/BalancerUserGuide2.pdf
Of course it impacts performance (it would even if it were automatic), but there’s a throttling parameter.

The same alarm handler that adds a Datanode could also invoke the rebalancer. The progress of the rebalance can be monitored programmatically, as described in the User Guide, so you don’t have to just guess when you could launch another one.

In addition, if you are using Ambari, you can trigger a rebalancing via Ambari:
http://docs.hortonworks.com/HDPDocuments/Ambari-2.0.1.0/bk_Ambari_Users_Guide/content/_how_to_rebalance_hdfs.html
or programmatically via the Ambari API, by invoking the Custom Command REBALANCEHDFS via a PUT List Requests

If you do it via Ambari, you’ll then be able to monitor its progress in the Ambari GUI, which could be useful. But if you’re not familiar with the Ambari API, I’m afraid that’s too complex a topic for a forum response.


Viewing all articles
Browse latest Browse all 3435


<script src="https://jsc.adskeeper.com/r/s/rssing.com.1596347.js" async> </script>