Design Twitter

Twitter (now X) is a microblogging platform where users post short messages called "tweets" to their followers. With ~400M daily active users generating hundreds of millions of tweets per day, it's a classic system design interview question that tests your ability to handle massive read-heavy workloads and real-time feed generation.

This walkthrough follows the Interview Framework and focuses on what you'd actually present in a 45-60 minute interview.

Phase 1: Requirements

Functional Requirements

1. Post tweets - Users can post text (280 chars), images, and videos
2. Follow/unfollow - Users can follow other users to see their tweets
3. Home timeline - Users see tweets from people they follow
4. User timeline - View a specific user's tweets
5. Like and retweet - Engage with tweets

Non-Functional Requirements

• Scale: 400M DAU, 200M tweets/day
• Latency: Home timeline < 200ms
• Availability: 99.99% (critical for breaking news)
• Consistency: Eventual consistency acceptable for feeds

Capacity Estimation

Users: 400M DAU
Tweets/day: 200M (avg 0.5 tweets/user/day)
Tweets/second: 200M / 86,400 ≈ 2,300 writes/sec

Timeline reads: 400M users × 10 views/day = 4B/day ≈ 46K reads/sec
Read:Write ratio: 46K / 2.3K ≈ 20:1 (heavily read-dominant)

Storage (5 years):
- Tweet text: 200M × 300 bytes × 365 × 5 ≈ 110 TB
- Media: 20M images/day × 200KB × 365 × 5 ≈ 7.3 PB

Phase 2: Data Model

Core Entities

User
- user_id: int64 (Snowflake ID)
- username: string (unique)
- email: string
- created_at: timestamp

Tweet
- tweet_id: int64 (Snowflake ID, time-sortable)
- author_id: int64
- content: string (280 chars; empty for pure retweet)
- retweet_of: int64 (nullable; original tweet_id)
- media_urls: string[] (resolved from uploaded media_ids)
- created_at: timestamp

Follow
- follower_id: int64
- followee_id: int64
- created_at: timestamp

Like
- user_id: int64
- tweet_id: int64
- created_at: timestamp

Retweets are modeled as tweets that reference the original via retweet_of. A quote tweet is just a retweet with non-empty content.

Why Snowflake IDs?

Twitter invented Snowflake IDs: 64-bit integers encoding timestamp + machine ID + sequence number.

| 1 bit: unused | 41 bits: timestamp | 10 bits: machine | 12 bits: sequence |

• 41 bits of timestamp → ~69 years of unique IDs
• 10 bits of machine ID → 1,024 machines
• 12 bits of sequence → 4,096 IDs per machine per millisecond

Benefits:

• Time-sortable: Can range-scan by time without secondary index
• Decentralized: No coordination needed to generate unique IDs
• Compact: 64 bits vs 128-bit UUIDs

Phase 3: API Design

REST APIs

POST /tweets
  Body: { content, media_ids[], retweet_of? }
  Response: { tweet_id, created_at }

GET /timeline/home?cursor=<tweet_id>&limit=20
  Response: { tweets[], next_cursor }

GET /timeline/user/:user_id?cursor=<tweet_id>&limit=20
  Response: { tweets[], next_cursor }

POST /users/:user_id/follow
DELETE /users/:user_id/follow

POST /tweets/:tweet_id/like
DELETE /tweets/:tweet_id/like

Phase 4: High-Level Design

Tweet Write Flow

1. User posts tweet via API Gateway
2. Tweet Service validates and stores tweet in Tweet Store
3. Publishes TweetCreated event to message queue
4. Fanout Service consumes event, checks author's follower count
5. If author has < 10K followers: inserts tweet_id into each follower's timeline cache
6. If author is a celebrity: skips fan-out (tweets fetched at read time via pull)

Timeline Read Flow

1. User requests home timeline
2. Timeline Service fetches tweet_ids from user's timeline cache (push results)
3. Queries Social Graph for user's followed celebrities, fetches their recent tweet_ids from Tweet Store (pull)
4. Merges both lists of tweet_ids, de-duplicates, sorts by tweet_id (reverse-chron)
5. Batch-fetches full tweet content, hydrates with author info, returns to client

User Timeline Read Flow

1. User requests a specific user's timeline
2. Query Tweet Store by author_id with a time-sorted index (e.g., author_id, tweet_id DESC)
3. Apply cursor (max tweet_id) and limit, then hydrate author info

Phase 5: Deep Dives

The Fan-out Problem

When a user posts a tweet, how do we deliver it to all followers? This is the core challenge of Twitter's design.

Approach 1: Fan-out on Write (Push Model)

When a tweet is posted, immediately write it to every follower's timeline cache.

User A posts tweet
→ Fetch A's 1000 followers
→ Insert tweet_id into 1000 timeline caches

Pros:

• Timeline reads are fast (pre-computed)
• Simple read path

Cons:

• Celebrity problem: User with 50M followers = 50M writes per tweet
• Write amplification is massive
• Wasted work for inactive users

Approach 2: Fan-out on Read (Pull Model)

When a user requests their timeline, fetch tweets from all users they follow.

User B requests timeline
→ Fetch B's 500 followees
→ Query recent tweets from each
→ Merge-sort and return top N

Pros:

• No write amplification
• No wasted work

Cons:

• Slow reads (must query many users)
• High read-time compute
• Cold start problem

Approach 3: Hybrid (What Twitter Actually Does)

Combine both approaches based on follower count:

if author.follower_count < 10,000:
    fan_out_on_write(tweet, followers)  # Push
else:
    mark_as_celebrity(author)  # Pull at read time

For timeline reads:

1. Fetch tweet_ids from timeline cache (push results)
2. Fetch recent tweet_ids from followed celebrities (pull)
3. Merge, de-duplicate, and sort by tweet_id (reverse-chron)
4. Batch-fetch full content

Timeline Cache Design

Key: timeline:{user_id}
Value: Sorted set of (tweet_id, score=tweet_id)
Max size: 800 tweets (older ones evicted)

Why Redis sorted sets?

• O(log N) insertion
• O(log N + M) range queries (M = results returned)
• Since Snowflake IDs are time-sortable, use tweet_id as score directly
• No need to store separate timestamps
• Reverse-chron timeline: fetch highest scores first

Handling Hot Tweets

A viral tweet can receive millions of likes/retweets in minutes. Naive counter increments would crush the database.

Solution: Sharded Counters

Instead of:
  tweet:123:likes = 5000000

Use:
  tweet:123:likes:shard:0 = 625000
  tweet:123:likes:shard:1 = 625000
  ...
  tweet:123:likes:shard:7 = 625000

• Write: Randomly pick a shard, increment
• Read: Sum all shards (can be cached)
• Reduce contention by 8x with 8 shards

Database Choices

Scaling the Social Graph

Storing follow relationships efficiently is critical.

Schema:

-- For "who do I follow?" queries
CREATE TABLE following (
    follower_id BIGINT,
    followee_id BIGINT,
    PRIMARY KEY (follower_id, followee_id)
);

-- For "who follows me?" queries (fanout)
CREATE TABLE followers (
    followee_id BIGINT,
    follower_id BIGINT,
    PRIMARY KEY (followee_id, follower_id)
);

Denormalized into two tables for efficient queries in both directions. Partitioned by the first column of primary key.

Extensions (Optional)

If time permits, mention a few add-ons beyond the core timeline:

• Notifications (fanout of mentions, replies, and likes)
• Trends (streaming aggregation + top-K by time window)
• Direct messages (separate service and storage)

Search Architecture (Optional)

Twitter needs near-real-time search (tweets searchable within seconds).

Architecture:

1. Tweet posted → Kafka event
2. Search Indexer consumes, tokenizes text
3. Writes to inverted index (term → tweet_ids)
4. Tiered index: real-time (seconds) + historical

Index structure:

Inverted Index:
  "breaking" → [tweet_id_1, tweet_id_2, ...]
  "news"     → [tweet_id_1, tweet_id_3, ...]

Query "breaking news":
  Intersect posting lists → Rank by relevance + recency

Twitter open-sourced their search engine called "Earlybird" (based on Lucene).

Common Pitfalls

Interview Checklist

Before ending your design, verify you've covered:

• Hybrid fan-out strategy (push for normal users, pull for celebrities)
• Timeline cache with sorted sets
• Snowflake IDs for tweets (time-sortable)
• Sharded counters for viral tweets
• Optional: search indexing pipeline (near-real-time)
• CDN for media delivery
• Discussed trade-off: eventual consistency for timeline
• Mentioned monitoring: timeline latency, fanout lag

Summary