RabbitMQ

How does RabbitMQ guarantee messages are not lost?

• Where can loss happen
  • publisher -> exchange
    • Producer confirmation mechanism
      • ack publish-confirm
      • nack publish-confirm
      • ack publish-return
      • How to handle a failed message
        • Callback method
        • Log it
        • Save to the database and resend on a schedule
  • exchange -> queue
    • Message persistence
      • Exchange persistence
      • Queue persistence
      • Message persistence
  • queue
  • queue -> consumer
    • Consumer acknowledgment mechanism

How is the duplicate-consumption problem in RabbitMQ solved?

Causes of duplicate consumption

• Network jitter
• The consumer crashed

Solutions

• Give each message a unique identifier id
• Idempotency schemes
  • Distributed lock
  • Database lock
    • Pessimistic lock
    • Optimistic lock

Dead-letter exchange (delay queue) in RabbitMQ

Delay queue = dead-letter exchange

Message backlog problem

1. Add more consumers
2. Enable a thread pool on the consumer side
3. Expand the queue capacity
   1. Lazy queue

RabbitMQ’s high-availability mechanism

1. Regular cluster
2. Mirrored cluster
3. Quorum queue

Kafka