SQS

• Messages can be stored in SQS for a short duration of time (maximum 14 days)

• Lambda provides event source mappings to capture sqs event

• If you delete a queue, you must wait at least 60 seconds before creating a queue with the same name.

  • AWS.SimpleQueueService.PurgeQueueInProgress: Only one PurgeQueue operation on ${QueueName} is allowed every 60 seconds.

• QueueUrl is not QueueName

  • sample QueueName: topicA
  • sample QueueUrl: https://sqs.ap-southeast-1.amazonaws.com/616879594671/topicA
  • aws provide sqs.getQueueUrl to get QueueUrl from QueueName

• no way to subscribe listener,only polling to receiveMessage is supported [pull] like kafka

• [message lag] If the number of messages in the queue is extremely small, you might not receive any messages in a particular ReceiveMessage response. If this happens, repeat the request.

SQS FIFO

By default, FIFO queues support up to 300 messages per second (including send, receive and delete)

• You can enable high throughput mode for FIFO, then 3000 messages per second

SQS dead-letter queue

• The dead-letter queue of a FIFO queue must also be a FIFO queue. Similarly, the dead-letter queue of a standard queue must also be a standard queue.

Setup

1. create a queue
2. create a dead-letter queue for an existing queue.
3. Configuring a dead-letter queue (console) for it. SQS does not create the DLQ automatically
4. Once you complete the debugging, you can use the dead-letter queue redrive capability to move the messages back to the source queue via console

Receive message wait time

Long polling helps reduce the cost of using Amazon SQS by eliminating the number of empty responses (when there are no messages available for a ReceiveMessage request) and false empty responses (when messages are available but aren't included in a response). If a receive request collects the maximum number of messages, it returns immediately and not waits for the polling to time out.

Visibility timeout

The visibility timeout begins when a message-queue returns a message. Message is invisible (locked) util timeout ends

• If the consumer fails to process and delete the message before the visibility timeout expires, the message becomes visible to other consumers.
• If a message must be received only once, your consumer must delete it within the duration of the visibility timeout.
• When working with FIFO queues, DeleteMessage operations will fail if the request is received outside of the visibility timeout window.
• If the visibility timeout is 0 seconds, the message must be deleted within the same millisecond it was sent, or it is considered abandoned

SNS

• Latency: 30ms

Kinesis

Unfortunately, while Kinesis can handle huge amounts of data, it does not auto-scale. Instead, we must explicitly set the throughput by defining the number of shards our stream will consist of, each shard providing a specified capacity.