I am trying to demo Kafka using the VirtualBox VM, sandbox version 2.4. I am able run the kafka-console-producer and kafka-console-consumer scripts and all works just fine. I am also able to create a producer on the windows host using the kafka-python client and that successfully delivered the message to the consumer running in the VM. And that is where success ends.
The goal of this exercise is to setup some Raspberry Pis that will submit events to a topic hosted on the VM, and that is where I am facing errors. Using the same code as used in the Windows host, I am not able to get the code on the Raspberry Pis to work. Below are the detailed errors. I installed nmap and confirmed that port 6667 is open from the Pis to the VM (and I believe to the broker). For the KafkaClient in the code on the Pi, I am using the IP address of the Windows host. Then in the Virtualbox VM I have a NAT entry to forward the data on port 6667 to 10.0.2.15 on port 6667.
When I try and run the code I get the following error…
pi@rp03 ~ $ python simpleProducerTest.py
No handlers could be found for logger “kafka.conn”
Traceback (most recent call last):
File “simpleProducerTest.py”, line 10, in <module>
producer.send_messages(‘motionAlert’,’sent from RP03′)
File “/usr/local/lib/python2.7/dist-packages/kafka/producer/simple.py”, line 54, in send_messages topic, partition, *msg
File “/usr/local/lib/python2.7/dist-packages/kafka/producer/base.py”, line 349, in send_messages return self._send_messages(topic, partition, *msg)
File “/usr/local/lib/python2.7/dist-packages/kafka/producer/base.py”, line 390, in _send_messages fail_on_error=self.sync_fail_on_error
File “/usr/local/lib/python2.7/dist-packages/kafka/client.py”, line 480, in send_produce_request (not fail_on_error or not self._raise_on_response_error(resp))]
File “/usr/local/lib/python2.7/dist-packages/kafka/client.py”, line 247, in _raise_on_response_error
raise resp
kafka.common.FailedPayloadsError
Any thoughts or ideas on how to resolve this?!?