How do load balance aync messages? - rnakidi/dsa GitHub Wiki
If you’re working with event-driven systems, here’s an interview question you must know.
“How can you load balance a huge number of asynchronous messages across consumers?”
You let the message consumers compete against each other. This is also known as the Competing Consumers Pattern.
How does it work?
[1] One or more producers add messages to a queue. These are basically tasks.
[2] Multiple consumer instances are set up to process messages or tasks from this queue.
[3] Each consumer competes to retrieve and process messages.
[4] Once a consumer successfully claims a message, it becomes unavailable to other consumers.
[5] After processing, the consumer acknowledges the message and removes it from the queue.
As you can see, the process is straightforward.
One important point is ensuring that a message is processed by only one consumer. In other words, how is the message claimed by a consumer made unavailable to other consumers?
Different platforms handle it in different ways.
✅ RabbitMQ
-
Consumers set a prefetch count, limiting the number of unacknowledged messages they can have.
-
When a consumer receives a message, it’s considered “in flight” and won’t be delivered to other consumers.
✅ Azure Service Bus uses a peek-lock mechanism
-
A consumer receives a message in peek-lock mode, which locks the message.
-
The message remains in the queue but is invisible to other consumers.
-
After processing, the consumer marks it as complete.
-
Once the lock expires, the message becomes visible again.
✅ AWS SQS sets a visibility timeout
-
When a consumer receives a message, SQS sets a visibility timeout.
-
During this timeout, the message is hidden from other consumers. After processing, the consumer deletes the message.
-
If the timeout expires before the message is deleted, it becomes visible again for other consumers.