I’m working with HDP2.2 on Centos6.5 as a single node cluster in an EC2 instance.
In order to make the Kafka Broker available remotely, the documentation instructs that you set host.name in the kafka server.properties file to the public facing FQDN (see reference).
The problem here is if I make that change and start the broker, Ambari seems to be resetting it to the server hostname (the internal FQDN of the EC2 server).
Now it would seem that in order for everything else to work normally, the /etc/hosts file/, hostname & /etc/sysconfig/network file all have to be set to the internal FQDN, which appears to be where Ambari is getting it from. Changing any of those gives the ‘Heartbeat lost’ issues in Ambari on server restart (as I would expect).
So how do I get around this? Presumably I’m not the first person to attempt to remote connect to a Kafka broker on EC2.
References:
Config:
[root@ip-172-31-32-129 ~]# hostname
ip-172-31-32-129.eu-west-1.compute.internal
[root@ip-172-31-32-129 ~]# cat /etc/sysconfig/network
NETWORKING=yes
HOSTNAME=ip-172-31-32-129.eu-west-1.compute.internal
NOZEROCONF=yes
NETWORKING_IPV6=yes
[root@ip-172-31-32-129 ~]# cat /etc/hosts
127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4
::1 localhost localhost.localdomain localhost6 localhost6.localdomain6
172.31.32.129 ip-172-31-32-129 ip-172-31-32-129.eu-west-1.compute.internal
52.16.107.90 myserver myserver.mydomain.com