Apache Kafka - kdevkr/mambo-box GitHub Wiki
๋๊ท๋ชจ ํธ๋ํฝ์ ์ฒ๋ฆฌํ๊ธฐ ์ํ ์ํคํ ์ฒ์๋ ์ Apache Kafka ๊ฐ ํฌํจ๋์ด ์์๊น?
๋์ปค ์ปดํฌ์ฆ๋ก Local Kafka ํ๊ฒฝ ๊ตฌ์ฑํ๊ธฐ
version: '3'
services:
kafka:
image: confluentinc/confluent-local:7.5.6
hostname: kafka
container_name: kafka
ports:
- "9092:9092"
- "8082:8082"
environment:
KAFKA_NODE_ID: 1
KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: 'CONTROLLER:PLAINTEXT,PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT'
KAFKA_ADVERTISED_LISTENERS: 'PLAINTEXT://kafka:29092,PLAINTEXT_HOST://localhost:9092'
KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1
KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0
KAFKA_TRANSACTION_STATE_LOG_MIN_ISR: 1
KAFKA_TRANSACTION_STATE_LOG_REPLICATION_FACTOR: 1
KAFKA_PROCESS_ROLES: 'broker,controller'
KAFKA_CONTROLLER_QUORUM_VOTERS: '1@kafka:29093'
KAFKA_LISTENERS: 'PLAINTEXT://kafka:29092,CONTROLLER://kafka:29093,PLAINTEXT_HOST://0.0.0.0:9092'
KAFKA_INTER_BROKER_LISTENER_NAME: 'PLAINTEXT'
KAFKA_CONTROLLER_LISTENER_NAMES: 'CONTROLLER'
KAFKA_LOG_DIRS: '/tmp/kraft-combined-logs'
KAFKA_REST_HOST_NAME: rest-proxy
KAFKA_REST_BOOTSTRAP_SERVERS: 'kafka:29092'
KAFKA_REST_LISTENERS: "http://0.0.0.0:8082"
CLUSTER_ID: '4L6g3nShT-eMCtK--X86sw'
kafka-ui:
container_name: kafka-ui
image: provectuslabs/kafka-ui:latest
ports:
- "8080:8080"
depends_on:
- kafka
environment:
KAFKA_CLUSTERS_0_NAME: local
KAFKA_CLUSTERS_0_BOOTSTRAPSERVERS: kafka:29092
DYNAMIC_CONFIG_ENABLED: 'true' # not necessary, added for tests
KAFKA_CLUSTERS_0_AUDIT_TOPICAUDITENABLED: 'true'
KAFKA_CLUSTERS_0_AUDIT_CONSOLEAUDITENABLED: 'true'
- confluent-local : Apache Kafkaยฎ in KRaft mode with zero configuration setup.
- kafka-ui : Web Management UI
๋๋ถ๋ถ wurstmeister/kafka-docker ๋๋ confluentinc/cp-kafka ๋ก Kafka ํด๋ฌ์ค๋ฌ๋ฅผ ๊ตฌ์ฑํ๋ค.
์ปจํ๋ฃจ์ธํธ ํ๋ซํผ ๊ตฌ์ฑ
https://github.com/confluentinc/cp-all-in-one/blob/7.5.0-post/cp-all-in-one/docker-compose.yml
ํ์ฉ ๋ฐ ์ด์ ๋ ธํ์ฐ
- Kafka ์ด์คํ๋ก ๋ค์ํ ์ฅ์ ์ํฉ ์๋ฒฝ ๋์ฒํ๊ธฐ
- ํด๋ผ์ฐ๋ ํ๊ฒฝ์์์ Kafka ์ด์๊ธฐ
- ์นดํ์นด, ๋๊ท๋ชจ ํด๋ฌ์คํฐ ์ด์ ํ๊ธฐ
- ์ ๋ขฐ์ฑ ์๋ ์นดํ์นด ์ ํ๋ฆฌ์ผ์ด์ ์ ๋ง๋๋ 3๊ฐ์ง ๋ฐฉ๋ฒ
- Kafka๋ฅผ ํ์ฉํ ์ด๋ฒคํธ ๊ธฐ๋ฐ ์ํคํ ์ฒ ๊ตฌ์ถ
- Kafka Streams๋ฅผ ํ์ฉํ ์ด๋ฒคํธ ์คํธ๋ฆผ ์ฒ๋ฆฌ ์ฝ์ง๊ธฐ
- ๋ค์ด๋ฒ ์ค์ผ์ผ๋ก ์นดํ์นด ์ปจ์๋จธ ์ฌ์ฉํ๊ธฐ
- ๋ฐ๋ฅด๊ฒ, ๋น ๋ฅด๊ฒ! Reactive๋ฅผ ํ์ Spring Kafka
- AWS MSK Connect ํจ๊ณผ์ ์ผ๋ก ์ด์ํ๊ธฐ
- Amazon MSK์์ ์ฌ๋ฐ๋ฅธ ํด๋ฌ์คํฐ ์ ํ์ ์ ํํ๋ ๋ฐฉ๋ฒ
- ๋ฐ์ดํฐ ์ฑํฌ๋ฅผ ์ํ MSK Connect ๋์
- B2B ์๋ฆผ ์๋น์ค์ Kafka CDC ์ ์ฉํ๊ธฐ
- Apache Kafka๋ฅผ ์ฌ์ฉํ์ฌ EDA ์ ์ฉํ๊ธฐ
- ์ค๋ฌด ๊ด์ ์์์ Apache Kafka ํ์ฉ
- ์ฐ๋ฆฌ ํ์ ์นดํ์นด๋ฅผ ์ด๋ป๊ฒ ์ฌ์ฉํ๊ณ ์์๊น
- ksqlDB๋ฅผ ํ์ฉํ ์ฆ๊ถ์ฌ์ ์ค์๊ฐ ๋ฐ์ดํฐ ์ฒ๋ฆฌํ๊ธฐ