How to Read This Post

Each scenario shows a diagram first, then a short note on why that system fits best. Complexity increases as you scroll.

System Strength
RabbitMQ Smart broker, routing flexibility, push-based delivery
Kafka Immutable log, high throughput, replay from any offset
JetStream NATS Lightweight persistence, built-in dedup, low operational cost

Level 1 — Foundations

1. Simple Task Queue

flowchart LR P[Producer] -->|task| Q[(Queue)] Q -->|deliver| C[Consumer] style P fill:#4a9eff,stroke:#2d7ed8,color:#fff style Q fill:#ff6b6b,stroke:#d44,color:#fff style C fill:#51cf66,stroke:#37b24d,color:#fff

RabbitMQ. Push-based delivery with acknowledgments. Consumer confirms completion before the broker removes the message. No polling, no offset tracking — just send and ack.

2. Broadcast Fan-out

flowchart LR P[Publisher] --> FE{Fanout\nExchange} FE --> Q1[(Queue A)] FE --> Q2[(Queue B)] FE --> Q3[(Queue C)] Q1 --> C1[Notifications] Q2 --> C2[Analytics] Q3 --> C3[Audit Log] style P fill:#4a9eff,stroke:#2d7ed8,color:#fff style FE fill:#ffd43b,stroke:#f59f00,color:#333 style Q1 fill:#ff6b6b,stroke:#d44,color:#fff style Q2 fill:#ff6b6b,stroke:#d44,color:#fff style Q3 fill:#ff6b6b,stroke:#d44,color:#fff style C1 fill:#51cf66,stroke:#37b24d,color:#fff style C2 fill:#51cf66,stroke:#37b24d,color:#fff style C3 fill:#51cf66,stroke:#37b24d,color:#fff

RabbitMQ. Fanout exchange copies every message to all bound queues. Each downstream service gets its own independent queue with separate ack tracking.

3. Fire-and-Forget Pub/Sub

flowchart LR P[Publisher] -->|"sensor.temperature"| NATS((NATS Core)) NATS --> S1[Subscriber A] NATS --> S2[Subscriber B] NATS --> S3[Subscriber C] style P fill:#4a9eff,stroke:#2d7ed8,color:#fff style NATS fill:#27ae60,stroke:#1e8449,color:#fff style S1 fill:#a29bfe,stroke:#6c5ce7,color:#fff style S2 fill:#a29bfe,stroke:#6c5ce7,color:#fff style S3 fill:#a29bfe,stroke:#6c5ce7,color:#fff

NATS Core. Sub-millisecond latency, no broker-side persistence, zero configuration. If a subscriber is offline, the message is gone. Perfect for telemetry, heartbeats, ephemeral events where loss is acceptable.

Level 2 — Distribution

4. Competing Consumers (Fair Dispatch)

flowchart LR P1[Producer] -->|tasks| Q[(Task Queue)] Q -->|"prefetch=1"| C1[Worker 1] Q -->|"prefetch=1"| C2[Worker 2] Q -->|"prefetch=1"| C3[Worker 3] style P1 fill:#4a9eff,stroke:#2d7ed8,color:#fff style Q fill:#ff6b6b,stroke:#d44,color:#fff style C1 fill:#51cf66,stroke:#37b24d,color:#fff style C2 fill:#51cf66,stroke:#37b24d,color:#fff style C3 fill:#51cf66,stroke:#37b24d,color:#fff

RabbitMQ. Round-robin dispatch across workers. prefetch=1 ensures a slow worker doesn’t hoard messages — the broker only sends the next task when the previous ack arrives.

5. Topic-Based Routing

flowchart LR P[Publisher] -->|"order.us.created"| TE{Topic\nExchange} TE -->|"order.us.*"| Q1[(US Orders)] TE -->|"order.*.created"| Q2[(All New Orders)] TE -->|"order.#"| Q3[(Order Archive)] Q1 --> C1[US Fulfillment] Q2 --> C2[Notification Service] Q3 --> C3[Data Warehouse] style P fill:#4a9eff,stroke:#2d7ed8,color:#fff style TE fill:#ffd43b,stroke:#f59f00,color:#333 style Q1 fill:#ff6b6b,stroke:#d44,color:#fff style Q2 fill:#ff6b6b,stroke:#d44,color:#fff style Q3 fill:#ff6b6b,stroke:#d44,color:#fff style C1 fill:#51cf66,stroke:#37b24d,color:#fff style C2 fill:#51cf66,stroke:#37b24d,color:#fff style C3 fill:#51cf66,stroke:#37b24d,color:#fff

RabbitMQ. Topic exchange with wildcard patterns (* matches one word, # matches zero or more). Routing logic lives in the broker — producers don’t need to know who’s listening.

6. Event Log with Replay

flowchart TB P1[Producer A] --> T P2[Producer B] --> T subgraph T["Topic: user-events"] direction LR P0["Partition 0
───────
offset 0 → 1 → 2 → 3 → 4 → 5"] P_1["Partition 1
───────
offset 0 → 1 → 2 → 3 → 4 → 5"] P_2["Partition 2
───────
offset 0 → 1 → 2 → 3 → 4 → 5"] end T --> CG1["Consumer Group A
📍 offset: 5 (live)"] T --> CG2["Consumer Group B
📍 offset: 2 (catching up)"] T --> CG3["New Consumer
📍 offset: 0 (full replay)"] style P1 fill:#4a9eff,stroke:#2d7ed8,color:#fff style P2 fill:#4a9eff,stroke:#2d7ed8,color:#fff style T fill:#1e3a8a,stroke:#e94560,color:#fff style P0 fill:#546e7a,stroke:#90a4ae,color:#fff style P_1 fill:#546e7a,stroke:#90a4ae,color:#fff style P_2 fill:#546e7a,stroke:#90a4ae,color:#fff style CG1 fill:#51cf66,stroke:#37b24d,color:#fff style CG2 fill:#ffd43b,stroke:#f59f00,color:#333 style CG3 fill:#ff6b6b,stroke:#d44,color:#fff

Kafka. The log is immutable — messages aren’t deleted on read. Each consumer group tracks its own offset. A new service can replay from offset 0 to rebuild its entire state. This is impossible with RabbitMQ’s destructive reads.

Level 3 — Reliability

7. Ordered Processing at Scale

flowchart TB subgraph Producers O1["Order: customer-7
key = customer-7"] O2["Order: customer-3
key = customer-3"] O3["Order: customer-7
key = customer-7"] end subgraph Kafka["Kafka Topic: orders"] direction LR PA["Partition 0
───────
customer-3 events
(hash mod 3 = 0)"] PB["Partition 1
───────
customer-7 events
(hash mod 3 = 1)"] PC["Partition 2
───────
(other customers)"] end O1 -->|"hash(key)"| PB O2 -->|"hash(key)"| PA O3 -->|"hash(key)"| PB subgraph CG["Consumer Group"] C1["Consumer 1 ↔ P0"] C2["Consumer 2 ↔ P1"] C3["Consumer 3 ↔ P2"] end PA --> C1 PB --> C2 PC --> C3 style O1 fill:#4a9eff,stroke:#2d7ed8,color:#fff style O2 fill:#4a9eff,stroke:#2d7ed8,color:#fff style O3 fill:#4a9eff,stroke:#2d7ed8,color:#fff style PA fill:#546e7a,stroke:#90a4ae,color:#fff style PB fill:#546e7a,stroke:#90a4ae,color:#fff style PC fill:#546e7a,stroke:#90a4ae,color:#fff style C1 fill:#51cf66,stroke:#37b24d,color:#fff style C2 fill:#51cf66,stroke:#37b24d,color:#fff style C3 fill:#51cf66,stroke:#37b24d,color:#fff

Kafka. Same key always maps to same partition. All events for customer-7 are processed in order by a single consumer. You scale horizontally by adding partitions — ordering is preserved per key, parallelism per partition.

8. Offline Consumer / Catch-Up

sequenceDiagram participant P as Publisher participant JS as JetStream participant C as Consumer P->>JS: publish(msg 1) P->>JS: publish(msg 2) P->>JS: publish(msg 3) Note over C: Consumer goes offline ⚡ P->>JS: publish(msg 4) P->>JS: publish(msg 5) Note over JS: Stream retains all
unacked messages Note over C: Consumer reconnects 🔄 JS->>C: deliver(msg 1) C->>JS: ack JS->>C: deliver(msg 2) C->>JS: ack JS->>C: deliver(msg 3) C->>JS: ack JS->>C: deliver(msg 4) C->>JS: ack JS->>C: deliver(msg 5) C->>JS: ack

JetStream NATS. Adds persistence on top of NATS Core. Streams retain messages; durable consumers resume from their last ack. Lighter than Kafka’s operational footprint — no ZooKeeper, no JVM, single binary.

9. Dead Letter Queue + TTL Retry

flowchart LR P[Producer] --> Q[(Main Queue)] Q --> C{Consumer} C -->|"✓ success"| Done["Done ✓"] C -->|"✗ fail (3x)"| DLX{Dead Letter\nExchange} DLX --> DLQ[(Dead Letter\nQueue)] DLQ --> R{Retry Logic} R -->|"after TTL"| WQ[(Wait Queue\nTTL: 30s)] WQ -->|"TTL expires"| Q style P fill:#4a9eff,stroke:#2d7ed8,color:#fff style Q fill:#ff6b6b,stroke:#d44,color:#fff style C fill:#ffd43b,stroke:#f59f00,color:#333 style Done fill:#51cf66,stroke:#37b24d,color:#fff style DLX fill:#e17055,stroke:#d63031,color:#fff style DLQ fill:#78909c,stroke:#546e7a,color:#fff style R fill:#ffd43b,stroke:#f59f00,color:#333 style WQ fill:#a29bfe,stroke:#6c5ce7,color:#fff

RabbitMQ. Dead Letter Exchanges (DLX) are a first-class feature. Failed messages route to a DLQ automatically. A wait queue with TTL creates retry loops — no external scheduler, no cron jobs. Kafka requires you to build all of this yourself.

Level 4 — Distributed Patterns

10. CQRS + Event Sourcing

flowchart TB CMD["Command API"] -->|"write"| WS[Write Service] WS -->|"append event"| KL["Kafka Event Log
(source of truth)"] KL --> PR1["Projector: SQL DB
📍 offset 1204"] KL --> PR2["Projector: Search Index
📍 offset 1198"] KL --> PR3["Projector: Analytics
📍 offset 1150"] PR1 --> DB1[(PostgreSQL)] PR2 --> DB2[(Elasticsearch)] PR3 --> DB3[(ClickHouse)] QR["Query API"] -->|"read"| DB1 QR -->|"search"| DB2 QR -->|"dashboard"| DB3 style CMD fill:#4a9eff,stroke:#2d7ed8,color:#fff style WS fill:#4a9eff,stroke:#2d7ed8,color:#fff style KL fill:#e94560,stroke:#c0392b,color:#fff style PR1 fill:#ffd43b,stroke:#f59f00,color:#333 style PR2 fill:#ffd43b,stroke:#f59f00,color:#333 style PR3 fill:#ffd43b,stroke:#f59f00,color:#333 style DB1 fill:#51cf66,stroke:#37b24d,color:#fff style DB2 fill:#51cf66,stroke:#37b24d,color:#fff style DB3 fill:#51cf66,stroke:#37b24d,color:#fff style QR fill:#a29bfe,stroke:#6c5ce7,color:#fff

Kafka. The event log IS the source of truth — not the database. Each projector consumes the same stream at its own pace to build a specialized read model. Need a new view? Deploy a new consumer, replay from offset 0. Existing projectors are unaffected.

11. Saga Choreography

flowchart LR subgraph Happy Path OS[Order Service] -->|"order.created"| K1((Kafka)) K1 --> PS[Payment Service] PS -->|"payment.completed"| K2((Kafka)) K2 --> IS[Inventory Service] IS -->|"stock.reserved"| K3((Kafka)) K3 --> SS[Shipping Service] SS -->|"shipment.scheduled"| K4((Kafka)) end subgraph Compensation PS2[Payment Service] -->|"payment.failed"| KC((Kafka)) KC --> OS2[Order Service] OS2 -->|"order.cancelled"| KC2((Kafka)) end style OS fill:#4a9eff,stroke:#2d7ed8,color:#fff style PS fill:#51cf66,stroke:#37b24d,color:#fff style IS fill:#ffd43b,stroke:#f59f00,color:#333 style SS fill:#a29bfe,stroke:#6c5ce7,color:#fff style PS2 fill:#51cf66,stroke:#37b24d,color:#fff style OS2 fill:#4a9eff,stroke:#2d7ed8,color:#fff style K1 fill:#e94560,stroke:#c0392b,color:#fff style K2 fill:#e94560,stroke:#c0392b,color:#fff style K3 fill:#e94560,stroke:#c0392b,color:#fff style K4 fill:#e94560,stroke:#c0392b,color:#fff style KC fill:#e94560,stroke:#c0392b,color:#fff style KC2 fill:#e94560,stroke:#c0392b,color:#fff

Kafka. No central orchestrator. Each service listens for events it cares about, does its work, emits the next event. Failures trigger compensating events that flow backwards. The event log provides a complete audit trail of every state transition.

12. Stream Processing Pipeline

flowchart LR SRC["Raw Events
(clickstream)"] --> T1["Topic:
raw-clicks"] T1 --> F["Filter &
Enrich"] F --> T2["Topic:
enriched-clicks"] T2 --> AGG["Windowed
Aggregation
(5 min tumbling)"] AGG --> T3["Topic:
click-counts"] T3 --> SINK1["Dashboard"] T3 --> SINK2["Alert Engine"] subgraph KS["Kafka Streams Application"] F AGG end style SRC fill:#4a9eff,stroke:#2d7ed8,color:#fff style T1 fill:#546e7a,stroke:#90a4ae,color:#fff style T2 fill:#546e7a,stroke:#90a4ae,color:#fff style T3 fill:#546e7a,stroke:#90a4ae,color:#fff style F fill:#ffd43b,stroke:#f59f00,color:#333 style AGG fill:#ffd43b,stroke:#f59f00,color:#333 style SINK1 fill:#51cf66,stroke:#37b24d,color:#fff style SINK2 fill:#51cf66,stroke:#37b24d,color:#fff style KS fill:#1e3a8a,stroke:#e94560,color:#fff

Kafka Streams. Processing runs inside your application — no separate cluster (unlike Flink/Spark). Intermediate results go back to Kafka topics. Stateful operations (windowed counts, joins) use local RocksDB stores backed by changelog topics for fault tolerance. Exactly-once semantics built in.

Level 5 — Expert

13. Multi-Region Replication

flowchart TB subgraph R1["Region: US-East"] P1[Producers] --> K1["Kafka Cluster"] K1 --> C1[Local Consumers] end subgraph R2["Region: EU-West"] P2[Producers] --> K2["Kafka Cluster"] K2 --> C2[Local Consumers] end K1 <-->|"MirrorMaker 2
async replication
offset translation"| K2 subgraph R3["Region: AP-Southeast"] K3["Kafka Cluster
(read replica)"] K3 --> C3[Local Consumers] end K1 -->|"MirrorMaker 2
one-way"| K3 style P1 fill:#4a9eff,stroke:#2d7ed8,color:#fff style P2 fill:#4a9eff,stroke:#2d7ed8,color:#fff style K1 fill:#e94560,stroke:#c0392b,color:#fff style K2 fill:#e94560,stroke:#c0392b,color:#fff style K3 fill:#78909c,stroke:#546e7a,color:#fff style C1 fill:#51cf66,stroke:#37b24d,color:#fff style C2 fill:#51cf66,stroke:#37b24d,color:#fff style C3 fill:#51cf66,stroke:#37b24d,color:#fff

Kafka. MirrorMaker 2 replicates topics across datacenters with automatic offset translation. US-East ↔ EU-West runs active-active (both produce and consume locally). AP-Southeast runs as a read replica. Consumers see local latency regardless of where the event originated.

14. Exactly-Once with Deduplication

sequenceDiagram participant P as Producer participant JS as JetStream participant C as Consumer P->>JS: publish(msg, Nats-Msg-Id: "order-123-v1") JS-->>P: ack (stored) Note over P: Network timeout,
producer retries ⚡ P->>JS: publish(msg, Nats-Msg-Id: "order-123-v1") JS-->>P: ack (duplicate, ignored) Note over JS: Dedup window: 2 min
Only one copy stored JS->>C: deliver(msg) C->>JS: ack Note over JS: Double-ack protection:
AckWait + MaxDeliver Note over C: Consumer processes
exactly once ✓

JetStream NATS. Built-in message deduplication using Nats-Msg-Id headers with a configurable time window. Producer retries are safe — duplicates are detected and discarded at the broker. Combined with MaxDeliver and AckWait on the consumer side, you get end-to-end exactly-once without idempotency logic in your application.

15. Hybrid Architecture: Right Tool Per Layer

flowchart TB CLIENT[Client] --> API[API Gateway] API <-->|"request-reply
< 1ms"| NATS((NATS)) subgraph Services["Microservices"] S1[Auth Service] S2[User Service] S3[Product Service] end NATS <--> Services Services -->|"domain events"| KAFKA((Kafka)) KAFKA --> STREAM["Stream Processing
(Kafka Streams)"] KAFKA --> ES["Event Store
(replay / audit)"] KAFKA --> PROJ["CQRS Projectors"] Services -->|"background jobs"| RMQ((RabbitMQ)) RMQ --> W1["Email Worker"] RMQ --> W2["PDF Generator"] RMQ --> W3["Image Resizer"] style CLIENT fill:#a29bfe,stroke:#6c5ce7,color:#fff style API fill:#a29bfe,stroke:#6c5ce7,color:#fff style NATS fill:#27ae60,stroke:#1e8449,color:#fff style S1 fill:#4a9eff,stroke:#2d7ed8,color:#fff style S2 fill:#4a9eff,stroke:#2d7ed8,color:#fff style S3 fill:#4a9eff,stroke:#2d7ed8,color:#fff style KAFKA fill:#e94560,stroke:#c0392b,color:#fff style STREAM fill:#ffd43b,stroke:#f59f00,color:#333 style ES fill:#ffd43b,stroke:#f59f00,color:#333 style PROJ fill:#ffd43b,stroke:#f59f00,color:#333 style RMQ fill:#ff6b6b,stroke:#d44,color:#fff style W1 fill:#51cf66,stroke:#37b24d,color:#fff style W2 fill:#51cf66,stroke:#37b24d,color:#fff style W3 fill:#51cf66,stroke:#37b24d,color:#fff

All three — each where it excels.

Layer System Why
Service-to-service RPC NATS Sub-millisecond request-reply, no serialization overhead
Event log / streaming Kafka Immutable log, replay, stream processing, multi-consumer
Background jobs RabbitMQ Ack-based delivery, DLQ, priority queues, mature tooling

No single messaging system wins every scenario. The right architecture uses each where its design assumptions match your requirements.

Decision Matrix

quadrantChart title Messaging System Sweet Spots x-axis "Simple Routing" --> "Complex Routing" y-axis "Ephemeral" --> "Durable / Replayable" quadrant-1 "Kafka" quadrant-2 "JetStream NATS" quadrant-3 "NATS Core" quadrant-4 "RabbitMQ" "Task Queue": [0.75, 0.35] "Fan-out": [0.70, 0.30] "Pub/Sub": [0.25, 0.15] "Topic Routing": [0.85, 0.40] "Event Log": [0.60, 0.85] "CQRS": [0.65, 0.90] "Saga": [0.55, 0.80] "Stream Processing": [0.70, 0.88] "DLQ Retry": [0.80, 0.45] "Catch-up Consumer": [0.35, 0.70] "Deduplication": [0.30, 0.75] "Request-Reply": [0.20, 0.20] "Multi-Region": [0.60, 0.95]

TL;DR

  • Need smart routing, task queues, or retry logic? → RabbitMQ
  • Need an event log, replay, or stream processing? → Kafka
  • Need lightweight persistence without operational overhead? → JetStream NATS
  • Need sub-millisecond fire-and-forget? → NATS Core
  • Building a real system? → Probably more than one