Pub/Sub

Publish-Subscribe (Pub/Sub) is a messaging pattern that enables asynchronous, one-to-many communication between services. Unlike point-to-point queues where each message goes to a single consumer, pub/sub broadcasts messages to all interested subscribers. In system design interviews, pub/sub appears whenever you need event-driven architectures, real-time notifications, or decoupled service communication.

This page covers what you need for interviews: how pub/sub differs from message queues, the core components and design patterns, push vs pull delivery, message filtering, and how to architect a scalable pub/sub system. Understanding these concepts helps you design systems that efficiently distribute events to multiple consumers without tight coupling.

Why Pub/Sub Matters

In distributed systems, services often need to communicate events to multiple consumers. Without pub/sub, you'd need direct connections between every producer and consumer—creating tight coupling and operational complexity.

With pub/sub, the producer publishes events to a topic, and all interested consumers subscribe to receive them automatically:

Key benefits of pub/sub:

• Decoupling — Publishers don't know who subscribes; subscribers don't know who publishes. Each can evolve independently.
• Scalability — Add new subscribers without modifying publishers. Scale publishers and subscribers independently.
• Flexibility — Multiple subscribers can process the same event differently (analytics, notifications, auditing).
• Real-time distribution — Events reach all subscribers immediately upon publication.
• Fault isolation — A slow or failed subscriber doesn't affect other subscribers or the publisher.

Pub/Sub vs Message Queues

Pub/sub and message queues are both asynchronous messaging patterns, but they serve different purposes. Understanding when to use each is critical for interviews.

Point-to-Point (Queue)

Each message is delivered to exactly one consumer. Multiple consumers compete for messages, but only one processes each.

Use cases: Task distribution, work queues, job processing where each task should be handled once.

Publish-Subscribe

Each message is delivered to all subscribers. Every subscriber receives a copy of the message.

Use cases: Event broadcasting, notifications, log aggregation, cache invalidation.

Comparison

Hybrid: Consumer Groups

Modern systems like Kafka blend both patterns. Consumer groups allow pub/sub with load balancing within each group:

• Each consumer group receives all messages (pub/sub behavior)
• Within a group, messages are distributed across workers (queue behavior)

This is the most common pattern in production systems—pub/sub for service-level fanout, queuing for worker-level load balancing.

Core Concepts

Publishers

Publishers are producers that send messages to topics. They're decoupled from subscribers—they publish events without knowing who will receive them.

Characteristics:

• Fire-and-forget (typically don't wait for subscriber acknowledgment)
• Can publish to multiple topics
• Don't manage subscriptions or delivery

Example events:

Topic: user.events
- user.created
- user.updated
- user.deleted

Topic: order.events
- order.placed
- order.shipped
- order.delivered

Topics

A topic is a named channel for messages of a particular type. Publishers write to topics; subscribers read from topics.

Topic design considerations:

• Granularity — Too broad (one topic for everything) loses filtering benefits. Too narrow (topic per event type) creates management overhead.
• Naming convention — Use hierarchical names: domain.entity.action (e.g., payments.transaction.completed)
• Schema — Define message schemas for each topic to ensure compatibility

Subscriptions

A subscription connects a subscriber to a topic. It defines:

• Which topic to receive messages from
• Delivery configuration (push or pull)
• Message filtering rules
• Acknowledgment deadline
• Retry policy

Subscription types:

Subscribers

Subscribers receive and process messages from their subscriptions. They must acknowledge messages to confirm processing.

Acknowledgment patterns:

• Automatic — Acknowledge upon receipt (risk: message lost if processing fails)
• Manual — Acknowledge after successful processing (safer, requires explicit ack)
• Negative ack (NACK) — Reject message, request redelivery

Push vs Pull Delivery

How messages reach subscribers is a fundamental design decision with significant trade-offs.

Push Model

The pub/sub system actively sends messages to subscribers as they arrive.

Pros:

• Low latency—subscribers receive messages immediately
• Simpler subscriber logic—no polling required
• Real-time delivery for time-sensitive events

Cons:

• Risk of overwhelming slow subscribers
• Requires endpoint availability (webhook URL)
• Needs rate limiting and backpressure handling

Technologies: AWS SNS, webhooks, WebSockets

Pull Model

Subscribers request messages at their own pace.

Pros:

• Natural backpressure—subscribers control consumption rate
• Tolerates subscriber downtime—messages wait in topic
• Better for batch processing

Cons:

• Higher latency (polling interval)
• Empty polls waste resources
• More complex subscriber logic

Technologies: Kafka, Google Pub/Sub (both modes), SQS

Comparison

Message Filtering

Not every subscriber needs every message. Filtering reduces unnecessary processing and network traffic.

Topic-Based Filtering

Subscribers choose which topics to subscribe to. Simple but coarse-grained.

Topics:
  orders.us.placed
  orders.us.shipped
  orders.eu.placed
  orders.eu.shipped

US Shipping Service subscribes to: orders.us.*
EU Analytics Service subscribes to: orders.eu.*

Pros: Simple, filtering happens at subscription level Cons: Requires many topics for fine-grained filtering, topic explosion

Attribute-Based Filtering

Messages carry attributes; subscriptions define filter expressions.

Message:
  topic: orders
  attributes:
    region: us
    type: placed
    priority: high

Subscription filter:
  region = "us" AND priority = "high"

Pros: Flexible filtering without topic proliferation Cons: More complex, filtering overhead on broker

Technologies: Google Pub/Sub (filter expressions), AWS SNS (filter policies), Kafka (consumer-side filtering)

Content-Based Filtering

Filter based on message body content.

Message body:
  {
    "order_id": "123",
    "total": 5000,
    "customer_type": "enterprise"
  }

Filter: total > 1000 AND customer_type = "enterprise"

Pros: Most flexible filtering Cons: Expensive—requires parsing message bodies, harder to optimize

Delivery Guarantees

Like message queues, pub/sub systems offer different delivery guarantees:

At-Least-Once (Default)

Most pub/sub systems default to at-least-once delivery. This means:

• Messages are never lost (if acknowledged by the broker)
• Messages may be delivered multiple times (network issues, subscriber crashes)
• Subscribers must be idempotent

Why duplicates happen:

1. Subscriber receives message
2. Subscriber processes message
3. Subscriber crashes before sending ACK
4. Pub/sub system redelivers message
5. Subscriber processes same message again

Designing for Idempotency

Since duplicates are inevitable with at-least-once delivery, design subscribers to handle them safely:

def handle_order_event(event):
    order_id = event.order_id

    # Check if already processed
    if db.exists(f"processed_events:{event.id}"):
        return  # Skip duplicate

    # Process the event
    process_order(event)

    # Mark as processed (idempotency key)
    db.set(f"processed_events:{event.id}", True, ttl=7_days)

Message Ordering

Pub/sub systems have different ordering guarantees:

Ordering strategies:

• No ordering — Simplest, highest throughput
• Per-key ordering — Events with same key delivered in order (e.g., all events for user_123)
• Global ordering — All events in order (limits parallelism)

High-Level Architecture

Let's design a scalable pub/sub system. This architecture appears in interviews when you're asked to design notification systems, event buses, or real-time data pipelines.

Components

Frontend Servers

• Accept publish requests from clients
• Validate messages (schema, size, permissions)
• Route to appropriate broker based on topic
• Handle subscription management API

Broker Cluster

• Store messages durably
• Manage topic partitions
• Track subscriber positions (offsets)
• Handle message delivery (push or pull)

Coordination Service

• Maintain cluster membership (which brokers are alive)
• Manage leader election for partitions
• Store topic and subscription metadata
• Handle distributed locks

Subscription Store

• Persist subscription configurations
• Store subscriber acknowledgment state
• Track delivery attempts and failures

Message Flow

Publishing:

1. Publisher sends message to frontend server
2. Frontend server validates and routes to broker
3. Broker writes to partition leader
4. Leader replicates to followers
5. Broker acknowledges publisher

Subscribing (Pull):

1. Subscriber requests messages from broker
2. Broker returns unacknowledged messages
3. Subscriber processes messages
4. Subscriber sends acknowledgments
5. Broker marks messages as delivered

Subscribing (Push):

1. Broker detects new message
2. Broker looks up subscription endpoints
3. Broker pushes to subscriber webhooks
4. Subscriber responds with acknowledgment
5. Broker retries on failure

Topic Partitioning

For high-throughput topics, a single broker becomes a bottleneck. Partitioning distributes load across multiple brokers.

Partition Assignment

Messages are assigned to partitions using a partition key:

partition = hash(partition_key) % num_partitions

Choosing partition keys:

• User ID — All events for a user go to same partition (ordering preserved)
• Order ID — All order events ordered
• Random/Round-robin — Maximum distribution, no ordering

Partition-Consumer Mapping

Each partition is consumed by one consumer in a consumer group:

3 partitions, 3 consumers → 1 partition each
3 partitions, 2 consumers → 1 consumer gets 2 partitions
3 partitions, 6 consumers → 3 consumers idle

Scaling Strategies

Scaling Publishers

Publishers are typically stateless—scale horizontally without coordination.

• Add more publisher instances behind load balancer
• Each instance can publish to any topic
• No coordination required between publishers

Scaling Subscribers

Pull-based: Add consumers to consumer group (up to partition count)

Push-based:

• Add more subscriber instances behind load balancer
• Pub/sub system distributes pushes across instances
• Implement rate limiting to prevent overwhelming subscribers

Scaling the Broker

Horizontal scaling:

• Add more brokers to the cluster
• Rebalance partitions across brokers
• Requires coordination service support

Partition scaling:

• Increase partition count for hot topics
• Warning: Can't easily decrease partitions
• Rebalancing causes brief consumer pauses

Handling Backpressure

When subscribers can't keep up:

Reliability and Fault Tolerance

Message Persistence

Messages should survive broker restarts:

• Write-ahead log (WAL) — Append messages to disk before acknowledging
• Replication — Copy to multiple brokers (typically 3 replicas)
• Retention policy — Keep messages for configurable duration (even after delivery)

Subscriber Failure Handling

When a subscriber fails mid-processing:

1. Acknowledgment timeout — If no ACK received, message requeued
2. Retry with backoff — Retry delivery with exponential backoff
3. Dead-letter topic — After max retries, move to DLT for investigation

Broker Failure Handling

With replication:

1. Leader fails — Follower promoted to leader
2. Follower fails — New replica created from leader
3. Metadata updated — Coordination service updates routing
4. Clients reconnect — Discover new leader automatically

Popular Technologies

Technology Comparison

When to Use Each

Kafka: High-throughput event streaming, need replay capability, event sourcing architectures.

Google Pub/Sub: GCP-native workloads, want managed service, need global distribution.

SNS + SQS: AWS-native, need reliable fanout with subscriber durability, microservices communication.

Redis Pub/Sub: Real-time features where message loss is acceptable (presence, typing indicators), need ultra-low latency.

Common Interview Patterns

Event-Driven Architecture

Pub/sub is the backbone of event-driven systems:

"When a user signs up, we publish a user.created event. The email service subscribes to send a welcome email, analytics subscribes to track signups, and the recommendation service subscribes to initialize user preferences. Each service is independent—we can add new subscribers without modifying the auth service."

Fanout Pattern

One event triggers multiple downstream actions:

Cache Invalidation

Broadcast cache invalidation events:

"When product data changes in the database, we publish to a cache.invalidate topic. All application servers subscribe and clear their local caches. This ensures eventual consistency across the fleet without direct server-to-server communication."

Log Aggregation

Collect logs from multiple services:

Real-Time Updates

Push updates to connected clients:

"For live sports scores, the score service publishes to a scores.{game_id} topic. WebSocket servers subscribe and push updates to connected clients. We can scale WebSocket servers independently—each subscribes to relevant game topics."

Common Pitfalls

1. Not Designing for Idempotency

Mistake: Assuming exactly-once delivery.

Reality: At-least-once is the norm. Duplicates happen due to network issues, retries, and rebalancing.

Fix: Design idempotent subscribers. Use event IDs to deduplicate. Make operations naturally idempotent (upsert vs insert).

2. Ignoring Message Ordering

Mistake: Assuming global message ordering across partitions.

Problem: User sees "order shipped" before "order placed" notification.

Fix: Use partition keys for related events. Accept that unrelated events may arrive out of order.

3. Missing Dead-Letter Handling

Mistake: No strategy for messages that repeatedly fail processing.

Problem: Poison messages block the queue or get lost after max retries.

Fix: Configure dead-letter topics. Monitor DLT size. Investigate and replay failed messages.

4. Over-Partitioning

Mistake: Creating hundreds of partitions "for future scale."

Problem: Increased memory overhead, longer rebalancing times, more broker connections.

Fix: Start with partitions = 3× expected consumer count. Add more when needed—can't easily reduce.

5. Tight Coupling via Message Contracts

Mistake: Subscribers depend on exact message structure. Any schema change breaks consumers.

Problem: Adding a field requires coordinated deployment across all subscribers.

Fix: Use schema evolution (Avro, Protobuf). Add optional fields. Subscribers ignore unknown fields.

6. No Backpressure Strategy

Mistake: Publishers produce faster than subscribers can consume indefinitely.

Problem: Memory exhaustion, message loss, system instability.

Fix: Set retention limits. Monitor consumer lag. Alert and scale before queues overflow.

Best Practices

Message Design

• Include event ID — For deduplication and debugging
• Include timestamp — When the event occurred (not when published)
• Include source — Which service/instance published
• Keep payloads small — Include IDs, not full objects. Subscribers fetch details if needed.
• Use schemas — Define and version message formats

{
  "event_id": "evt_abc123",
  "event_type": "order.placed",
  "timestamp": "2024-01-15T10:30:00Z",
  "source": "order-service-prod-1",
  "data": {
    "order_id": "ord_xyz789",
    "customer_id": "cust_456",
    "total_cents": 5999
  }
}

Topic Design

• One topic per event type — Not one topic for all events
• Use hierarchical naming — domain.entity.action (e.g., payments.refund.requested)
• Separate environments — prod.orders.placed vs staging.orders.placed

Subscription Design

• One subscription per use case — Analytics and notifications should be separate subscriptions
• Set appropriate acknowledgment deadlines — Long enough for processing, short enough for timely retry
• Configure retry policies — Exponential backoff with max attempts

Monitoring

Quick Reference

Pattern Selection

Delivery Model Selection

Interview Checklist

When discussing pub/sub:

• Explained why pub/sub over direct calls (decoupling, fanout)
• Distinguished from message queues (one-to-many vs one-to-one)
• Chose push vs pull delivery with reasoning
• Discussed message ordering requirements
• Specified delivery guarantee (at-least-once default)
• Mentioned subscriber idempotency
• Addressed filtering strategy (topic vs attribute-based)
• Discussed partitioning for scalability
• Mentioned dead-letter handling
• Named specific technology (Kafka, SNS, Google Pub/Sub)

What Interviewers Look For

1. Understanding the pattern — You know pub/sub enables one-to-many decoupled communication, distinct from point-to-point queues.

2. Appropriate application — You correctly identify when pub/sub fits (event fanout, notifications) vs when queues are better (task distribution).

3. Delivery semantics — You understand at-least-once delivery and design idempotent subscribers without being prompted.

4. Scalability thinking — You mention partitioning, consumer groups, and understand the relationship between partitions and parallelism.

5. Practical knowledge — You name real technologies (Kafka, SNS, Google Pub/Sub) and know their trade-offs rather than speaking abstractly.

6. Reliability awareness — You address failure scenarios: dead letters, acknowledgment timeouts, broker failures.

Pub/sub is infrastructure that enables event-driven architectures. Demonstrate you understand the pattern quickly, make appropriate technology choices, and focus on the interesting aspects of your specific design—the business logic and data flow that pub/sub enables.