002. Kafka Theory. Brokers and Topics - MarkHuntDev/my-kafka-exercises GitHub Wiki

Brokers

  • A Kafka cluster is composed of multiple brokers (servers)
  • Each broker is identified with its ID (integer)
  • Each broker contains certain topic partitions
  • After connecting to any broker (called a bootstrap broker), you will be connected to the entire cluster
  • A good number to get started is 3 brokers, but some big clusters have over 100 brokers
  • In these examples we choose to number brokers starting at 100 (arbitrary)

When you create a topic, Kafka will automatically assign the topic and distribute it across all your brokers.

Brokers and topics