Design WhatsApp

WhatsApp is a real-time messaging application that connects billions of users worldwide, handling 100+ billion messages daily. This question tests your understanding of WebSocket connections, message delivery guarantees, and scaling real-time systems.

This walkthrough follows the Interview Framework and focuses on what you'd actually present in a 45-60 minute interview.

Phase 1: Requirements (~5 minutes)

A messaging app has countless features, but your interviewer doesn't want you to cover them all. Clarify scope quickly to leave time for the interesting parts.

Functional Requirements

Frame these as user capabilities:

• One-on-one messaging — Users can send text messages to individuals
• Group messaging — Users can create groups (up to 100-200 participants) and send messages
• Message acknowledgment — Users see sent, delivered, and read status
• Media sharing — Users can send images, videos, and documents
• Offline support — Users receive messages sent while they were offline (up to 30 days)

Non-Functional Requirements

These define how well the system performs:

Ask clarifying questions:

• "Should we prioritize availability or consistency?" — Messages should arrive in order even if it takes slightly longer. Favor consistency within a conversation.
• "How long do we store messages on the server?" — WhatsApp stores messages only until delivered (30-day max for offline users). This is a key privacy feature.
• "Do we need to support multiple devices per user?" — Yes, many users have phone + desktop. This adds complexity.

Capacity Estimation

Do a quick back-of-envelope calculation to inform your design:

Given: 2B users, 100B messages/day, assume 20% peak concurrency.

Assumptions: avg text payload 200 bytes (encrypted content + metadata), 2% of messages include media at 300 KB avg, media is delivered via CDN.

Messages per second:

100B messages/day ÷ 86,400 sec ≈ 1.2 million messages/sec

Concurrent connections:

2B users × 20% online at peak ≈ 400M concurrent connections

Storage (text messages only, 30-day retention):

100B messages/day × 200 bytes avg × 30 days = 600 TB

Bandwidth (text only):

100B messages × 200 bytes ÷ 86,400 sec ≈ 231 MB/sec ≈ 2 Gbps

Media bandwidth (order of magnitude):

100B messages × 2% with media × 300 KB ≈ 600 TB/day

Media traffic dominates bandwidth, which is why uploads/downloads are pushed through CDN instead of the WebSocket path.

Phase 2: Data Model (~5 minutes)

Identify the key entities before jumping into APIs. This establishes shared vocabulary with your interviewer.

Core Entities

User
├── user_id (phone number or UUID)
├── name
├── avatar_url
└── last_seen

Conversation (Chat)
├── conversation_id
├── type (1:1 or group)
├── name (for groups)
├── created_at
└── metadata

ConversationParticipant
├── conversation_id
├── user_id
└── joined_at

Message
├── message_id
├── conversation_id
├── sender_id
├── content (encrypted)
├── type (text, image, video, etc.)
└── timestamp

MessageReceipt
├── message_id
├── recipient_id (user_id or client_id)
├── status (delivered, read)
└── updated_at

Client (Device)
├── client_id
├── user_id
├── device_type
└── push_token

Phase 3: API Design (~5 minutes)

Unlike most APIs where REST is appropriate, a chat app needs bidirectional real-time communication. This is a perfect use case for WebSockets.

Why WebSockets?

For messaging, both parties need to push data: the client sends messages, the server pushes incoming messages. WebSockets allow this over a single persistent connection, avoiding the overhead of repeated HTTP handshakes.

WebSocket Commands

Client → Server (Commands Sent):

// Create a conversation
{
  "action": "create_conversation",
  "participants": ["user_id_1", "user_id_2"],
  "type": "group",
  "name": "Family Chat"
}

// Send a message
{
  "action": "send_message",
  "conversation_id": "conv_123",
  "content": "Hello!",
  "type": "text",
  "client_message_id": "local_456"  // For deduplication
}

// Acknowledge message received
{
  "action": "ack",
  "message_id": "msg_789"
}

// Mark messages as read
{
  "action": "read_receipt",
  "conversation_id": "conv_123",
  "last_read_message_id": "msg_789"
}

Server → Client (Commands Received):

// New message notification
{
  "event": "new_message",
  "message_id": "msg_789",
  "conversation_id": "conv_123",
  "sender_id": "user_456",
  "content": "Hello!",
  "timestamp": "2024-01-15T10:30:00Z"
}

// Delivery/read status update
{
  "event": "status_update",
  "message_id": "msg_789",
  "status": "delivered",
  "timestamp": "2024-01-15T10:30:01Z"
}

REST Endpoints (Supporting APIs)

Some operations work better as REST:

// Upload media (returns attachment ID to include in message)
POST /api/v1/media
Content-Type: multipart/form-data

Response: { "media_id": "media_123", "url": "https://cdn.../media_123" }

// Fetch conversation history (pagination)
GET /api/v1/conversations/{id}/messages?before={timestamp}&limit=50

// Get user's conversations
GET /api/v1/conversations

Phase 4: High-Level Design (~15-25 minutes)

This is the core of your interview. Start with a working design, then evolve it.

Initial Architecture

Start with the components needed to satisfy requirements:

Components:

1. Load Balancer (L4) — TCP-level load balancing for WebSocket connections
2. WebSocket Servers — Maintain persistent connections with clients
3. Pub/Sub (Redis) — Route messages between WebSocket servers
4. Message Service — Business logic for message processing
5. Message DB — Durable storage for messages (Cassandra or similar)
6. Inbox Queue — Per-device queue for offline message delivery
7. Asset Service — Handles media uploads/downloads
8. Blob Storage + CDN — Store and serve media files

Message Flow: Sending a Message

Walk through the data flow as you draw:

"When User A sends a message to User B, the message hits A's WebSocket server and is forwarded to the Message Service. The service stores the message, adds it to B's device inboxes, then publishes to Redis Pub/Sub. B's WebSocket server receives the publish, finds B's connection, and delivers the message. When B's client acknowledges receipt, we remove the message from the inbox."

Step by step:

1. User A sends message via WebSocket to their connected server (WS1)
2. WS1 validates the message and forwards it to Message Service
3. Message Service generates a message_id and writes to Message DB (durable storage)
4. Message Service adds the message to each of User B's device inboxes (for delivery tracking)
5. Message Service publishes to Redis Pub/Sub channel for User B
6. If User B is online:
   - WS2 (B's server) receives the publish
   - WS2 delivers message to User B via WebSocket
   - User B's client sends ACK
   - WS2 removes message from that device's Inbox and publishes a status update
7. WS1 delivers the status update to User A
8. If User B is offline:
   - Message stays in Inbox
   - When B connects, their WS server reads Inbox and delivers

The Routing Problem

With hundreds of WebSocket servers, how does WS1 (sender's server) route a message to WS2 (receiver's server)?

Option 1: Connection Registry (Good)

Maintain a mapping of user_id → server_id in Redis:

user_123 → ws-server-47
user_456 → ws-server-12

When routing, look up the user's server and send directly. Update the registry when connections open/close.

Cons: Registry lookups add latency; stale entries if servers crash.

Option 2: Pub/Sub (Better)

Each WebSocket server subscribes to channels for its connected users. When User B connects to WS2, WS2 subscribes to channel user:B. To send a message to B, publish to that channel.

WS1 publishes: PUBLISH user:B "{message}"
WS2 (subscribed to user:B) receives and delivers

Pros: No registry to maintain; naturally handles server failures (re-subscribe on reconnect).

Option 3: Consistent Hashing (Alternative)

Hash user_id to determine which server handles their connection. Clients connect to their assigned server directly.

Cons: Requires client-side logic; rebalancing on server add/remove is complex.

Group Message Flow

Groups add complexity because a single message must reach N recipients:

1. User A sends message to Group (100 members)
2. WS1 forwards to Message Service; message stored in Message DB
3. Message Service adds message to inboxes of ALL 100 members (per device)
4. Message Service publishes to Redis for each member (user:<id>)
5. WS servers with connected members receive and deliver
6. Each member's client ACKs independently
7. Inbox entries cleared as ACKs arrive

For large groups, the "add to N inboxes" step can be offloaded to a background worker via a message queue (Kafka) to avoid blocking the sender.

Media Message Flow

Media files are handled separately to keep WebSocket servers lightweight:

1. Client uploads media to Asset Service via HTTPS
2. Asset Service stores in Blob Storage (S3)
3. Asset Service returns media_id and CDN URL
4. Client sends message via WebSocket with media_id reference
5. Recipient receives message with media_id
6. Recipient downloads media from CDN

Database Choices

Message Storage (Cassandra recommended):

• Write-heavy workload (1.2M writes/sec)
• Time-series data (messages ordered by timestamp)
• Need horizontal scaling
• No complex queries needed

-- Cassandra table design
CREATE TABLE messages (
    conversation_id UUID,
    message_id TIMEUUID,  -- Sortable by time
    sender_id UUID,
    content BLOB,         -- Encrypted
    media_id UUID,
    created_at TIMESTAMP,
    PRIMARY KEY (conversation_id, message_id)
) WITH CLUSTERING ORDER BY (message_id DESC);

Partition by conversation_id keeps all messages in a conversation together for efficient retrieval.

User/Conversation Metadata (PostgreSQL or DynamoDB):

• Lower volume, mostly reads
• Need for transactions (creating groups)
• Simpler access patterns

Connection State (Redis):

• Online/offline presence (optional user_id → server_id mapping for metrics/fallback)
• Last-seen or activity heartbeat tracking
• Very fast reads needed

Phase 5: Scaling & Trade-offs (~15-20 minutes)

With a working design in place, address the non-functional requirements and potential bottlenecks.

Scaling WebSocket Connections

Challenge: 400M concurrent connections across our fleet.

With careful optimization, a single server can handle 1-2 million concurrent connections (WhatsApp famously achieved this with Erlang). At 2M connections/server:

400M connections ÷ 2M per server = 200 WebSocket servers

Key optimizations:

• Use efficient event loops (epoll on Linux)
• Minimize per-connection memory overhead
• Connection pooling for downstream services

Scaling Message Throughput

Challenge: 1.2M messages/sec, with group fan-out multiplying this.

Strategies:

1. Batch writes to Cassandra — Buffer messages briefly and write in batches
2. Async inbox updates — Use Kafka to queue inbox updates for background processing
3. Shard Redis Pub/Sub — Use Redis Cluster with channels sharded by user_id

Handling Offline Users

Messages for offline users accumulate in their Inbox. When they reconnect:

1. Client connects and provides last_message_id for each conversation
2. Server queries Inbox for all pending messages
3. Server delivers messages in order
4. Client ACKs after persisting locally
5. Server clears Inbox entries

Preventing Inbox explosion: Set a maximum inbox size. If exceeded, delete oldest undelivered messages. WhatsApp's 30-day limit handles this.

Message Ordering

Users expect messages in a conversation to appear in the order they were sent. This is harder than it sounds in a distributed system.

Within a conversation: Use message_id based on server timestamp + sequence number. Cassandra's TIMEUUID provides ordering.

Across devices: If Alice sends from her phone, then immediately from desktop, both messages should appear in the same order for Bob. Use a per-conversation sequence number incremented atomically (Redis INCR).

Read Receipts at Scale

When User A reads a message from User B, A's read receipt must reach B. For group messages, N users might send read receipts for the same message.

Optimization: Don't send read receipts individually. Batch them:

"User A read messages up to msg_789 in conv_123"

The sender's client can derive which of their messages were read from this single update.

Storage note: Keep delivery/read state per recipient (e.g., MessageReceipt). For large groups, store a per-user last_read_message_id per conversation and avoid per-message receipts except for small groups or recent messages.

Multi-Device Support

Users with multiple devices need messages delivered to all of them:

1. Track clients separately from users
2. Inbox is per-client, not per-user
3. When sending a message to User B, add to Inbox of all B's clients
4. Each client ACKs independently

For UI status, aggregate per-device receipts into per-user status (e.g., delivered if any device ACKs; read when any device reports read).

End-to-End Encryption (High-Level)

WhatsApp uses the Signal Protocol. Key points for interviews:

• Each user generates public/private key pairs
• Messages encrypted with recipient's public key
• Server cannot read message content
• Keys distributed via the server; users can verify fingerprints out-of-band (QR code)

Trade-offs Discussion

Consistency vs Availability:

Recommendation: Favor consistency within a conversation (users notice out-of-order messages), but accept eventual consistency across conversations.

Latency vs Durability:

Recommendation: Synchronous write to durable storage before confirming to sender. The extra latency is acceptable; losing messages is not.

High Availability

Handling WebSocket server failure: If a server crashes, clients reconnect to another server via the load balancer. They fetch pending messages from Inbox on reconnect.

Monitoring & Observability

Mention these proactively:

• Message delivery latency (p50, p99)
• Connection count per server
• Inbox size distribution (large inboxes indicate offline users or delivery issues)
• Pub/Sub lag (messages waiting to be delivered)
• Error rates by message type

Common Pitfalls

Interview Checklist

Before wrapping up, verify you've covered:

Requirements Phase

• 4-5 functional requirements identified
• Scale, latency, availability clarified
• Quick capacity estimation completed (messages/sec, connections)

Data Model

• Key entities: User, Conversation, Message, Client, MessageReceipt
• Conversation-Participant relationship explained

API Design

• WebSocket choice justified (bidirectional, real-time)
• Key commands defined (send, receive, ack)
• Media handled separately via REST

High-Level Design

• Architecture diagram with data flow
• Message routing explained (Pub/Sub)
• Offline delivery handled (Inbox pattern)
• Media path separate from text

Scaling & Trade-offs

• Connection scaling addressed
• Consistency vs availability trade-off discussed
• At least one deep dive completed (routing, ordering, or groups)

Summary

The WhatsApp design question tests your understanding of real-time systems, message delivery guarantees, and scaling persistent connections. Focus on the Pub/Sub routing and Inbox pattern for delivery guarantees—these are the interesting parts that differentiate strong candidates.