004. Kafka Theory. Producers and Message Keys - MarkHuntDev/my-kafka-exercises GitHub Wiki
Producers
- Producers write data to topics (which is made of partitions)
- Producers automatically know to which broker and partition to write to
- In case of Broker failures, Producers will automatically recover
- Producers can choose to receive acknowledgment of data writes:
- acks=0: Producer won't wait for acknowledgement (possible data loss)
- acks=1: Producer will wait for leader acknowledgement (limited data loss)
- acks=all: Leader + replicas acknowledgment (no data loss)
Producers: Message keys
- Producers can choose to send a key with the message (string, number, etc...)
- If key=null, data is sent round robin (broker 101 then 102 then 103...)
- If a key is sent, then all messages for that key will always go to the same partition
- A key is basically sent if you need message ordering for a specific field (ex: truck_id)