We have several kafka clusters. We would like to maximize the throughput on our network by having the producers write to one interface but all replication and communication between nodes to be on a different interface. Each data node has two interfaces (eth0, eth1). We are getting near network saturation as the interfaces are only 1GB. What we would like to do is have eth0 be the interfaces that all topics are being written to and for eth1 to used for all replication between data nodes. Has anyone tried to do this, if so how was it accomplished? Is it possible to set the host.name file to the interface we want the kafka nodes to push replication across and then set the advertised.port so that zookeeper broadcasts the other interface for the producers to write to?
↧