Local Kafka with Docker - Gepardec/Hogarama GitHub Wiki
Running a minimal local Kafka with Docker
Note: This setup is following this guide: Guide to Setting Up Apache Kafka Using Docker | Baeldung
Single Node
To run a local Kafka installation with docker, we need to start a Zookeeper server and also an Apache Kafka server.
docker-compose.yml Configuration
The Zookeeper server needs to start first, so we configure this dependency in a docker-compose.yml file. This ensures that the Zookeeper server starts before the Kafka server and also stops after it.
Create a docker-compose.yml file with two services (zookeeper and kafka)
version: '2'
services:
zookeeper:
image: confluentinc/cp-zookeeper:6.0.3
environment:
ZOOKEEPER_CLIENT_PORT: 2181
ZOOKEEPER_TICK_TIME: 2000
ports:
- 22181:2181
kafka:
image: confluentinc/cp-kafka:6.0.3
depends_on:
- zookeeper
ports:
- 29092:29092
environment:
KAFKA_BROKER_ID: 1
KAFKA_ZOOKEEPER_CONNECT: zookeeper:2181
KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://localhost:29092
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT
KAFKA_INTER_BROKER_LISTENER_NAME: PLAINTEXT
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
Docker Image versions
For the Zookeeper and Kafka we use the image tags 6.0.3
due we use Kafka in version 2.6
and this tag of the Confluent-Image is compatile with this Kafka-version.
For more information see: Supported Versions and Interoperability | Confluent
Used Ports
Zookeeper
- Inside container: listening on port 2181 for the kafka service.
- Client running on the host: port 22181.
Kafka
- Inside container: port 9092
- Client running on the host: 29092
Starting the Kafka server
docker-compose up -d
The Kafka server should now be available on Port 29092.
Connecting to Kafka with KafDrop
To connect to the Kafka running in Docker, there is the possibility to use Kafdrop, an UI Tool for Kafka.
- Clone the repo
- Build it with
mvn clean package
- Run it with the following command:
java -jar ./target/kafdrop-2.1.0.jar --zookeeper.connect=localhost:22181 --kafka.brokers=localhost:29092
Note: Kafdrop will show an error if there are no topics defined on the Kafka. (So maybe push some messages with the Mock-Cli first)