Setting up Kafka
Important
GitHub Wiki is just a mirror of our online documentation.
We highly recommend using our website docs due to Github Wiki limitations. Only some illustrations, links, screencasts, and code examples will work here, and the formatting may be broken.
Please use https://karafka.io/docs.
Source: Kafka on Rails: Using Kafka with Ruby on Rails — Part 2 — Getting started with Rails and Kafka
Before we combine Kafka with Ruby, it would be good to have a workable local Kafka process. The easiest way to do that is by using our docker-compose.yml
present in Karafka:
git clone git@github.com:karafka/karafka.git
cd karafka
docker-compose up
To check that it works, you can just telnet to it:
telnet 127.0.0.1 9092
Trying 127.0.0.1...
Connected to 127.0.0.1.
Escape character is '^]'.
The docker-compose.yml
we provide with Karafka has a setting called KAFKA_ADVERTISED_HOST_NAME
, and it is by default set to localhost
.
Modify the KAFKA_ADVERTISED_HOST_NAME
to match your docker host IP if you want to connect to Kafka from other docker containers:
# KAFKA_ADVERTISED_HOST_NAME: localhost
KAFKA_ADVERTISED_HOST_NAME: 192.168.0.5
Once you've changed that, you should be able to connect from other docker containers to your Kafka by using the host IP address:
# Run an example docker container to check it via telnet
docker run -it --rm --entrypoint=bash python:3.8-slim-buster
# And then from within
apt update && apt install telnet
telnet 192.168.0.5 9092
Trying 192.168.0.5...
Connected to 192.168.0.5.
Escape character is '^]'.
Home | Getting started | Components | Producing Messages | Consuming messages | Concurrency and multithreading | Error handling and back off policy
This wiki is tracked by git and publicly editable. You are welcome to fix errors and typos. Any defacing or vandalism of content will result in your changes being reverted and you being blocked.