Kafka Insallation Guide - ajay3003/doc GitHub Wiki

To create an installation guide wiki page based on the provided commands, you can follow this outline:


Kafka and Zookeeper Setup Using Podman

This guide explains how to set up Apache Kafka and Zookeeper using Podman containers. It includes the process for pulling the necessary Docker images, setting up networks, and running the containers for both Kafka and Zookeeper.

Prerequisites

Ensure that you have Podman installed on your system. If not, install it by following the [official installation guide](https://podman.io/getting-started/installation).

Steps

1. Pull Kafka and Zookeeper Images

Begin by pulling the latest images for Kafka and Zookeeper from Docker Hub:

podman pull confluentinc/cp-zookeeper:latest
podman pull confluentinc/cp-kafka:latest

2. Clean Up Existing Containers (if any)

If you have any existing Kafka or Zookeeper containers running, stop and remove them:

podman stop kafka zookeeper
podman rm kafka zookeeper
podman network rm kafka-net

3. Create a New Network for Kafka and Zookeeper

Create a new network for Kafka and Zookeeper to communicate:

podman network create kafka-net

4. Start Zookeeper Container

Now, start the Zookeeper container. This container will be used by Kafka for distributed coordination:

podman run -d --name=zookeeper --network kafka-net -p 2181:2181 -e ZOOKEEPER_CLIENT_PORT=2181 -e ZOOKEEPER_TICK_TIME=2000 confluentinc/cp-zookeeper:latest

5. Start Kafka Container

Once Zookeeper is running, start the Kafka container. Make sure that Kafka can connect to the Zookeeper instance:

podman run -d --name=kafka --network kafka-net -p 9092:9092 -e KAFKA_BROKER_ID=1 -e KAFKA_ZOOKEEPER_CONNECT=zookeeper:2181 -e KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://localhost:9092 -e KAFKA_LISTENER_SECURITY_PROTOCOL_MAP=PLAINTEXT:PLAINTEXT -e KAFKA_INTER_BROKER_LISTENER_NAME=PLAINTEXT confluentinc/cp-kafka:latest

6. Create a Kafka Topic

To test the Kafka setup, create a test topic:

podman exec -it kafka kafka-topics --create --topic test-topic --bootstrap-server localhost:9092 --partitions 1 --replication-factor 1

7. Verify the Setup

To verify that Kafka is running and the topic has been created, list all topics:

podman exec -it kafka kafka-topics --list --bootstrap-server localhost:9092

This should display the created test-topic.


Conclusion

You have successfully set up Apache Kafka and Zookeeper using Podman containers. You can now start producing and consuming messages with Kafka.


Feel free to adjust the instructions for your wiki's style or add more detailed troubleshooting sections if necessary!