Numbers to Know

In system design interviews, you'll estimate storage requirements, bandwidth needs, and whether a particular architecture can handle the load. Having the right numbers in your head—and knowing how to use them—separates candidates who can design real systems from those who memorize patterns.

One of the biggest red flags is when candidates propose complex solutions based on outdated assumptions. They worry about database sizes that modern instances handle easily, or suggest sharding for workloads a single server manages comfortably. Many textbooks and courses teach numbers from 2015 or earlier—hardware has improved dramatically since then.

This page covers the essential numbers for 2026 and teaches you how to apply them in back-of-envelope calculations.

Latency Numbers Every Engineer Should Know

These foundational latencies help you reason about where time goes in your system. The key is understanding orders of magnitude—you don't need to memorize exact values.

Memory Hierarchy

Network Latency

Key insight: Within a datacenter, network round-trips (~0.5ms) are in the same order of magnitude as SSD access (~0.1ms)—both are sub-millisecond. Cross-region calls at 50-150ms are expensive—minimize cross-region hops in your design.

Data Operations

Modern Hardware Limits (2026)

Understanding what a single machine can do prevents premature optimization and over-engineering.

Memory

Implication: You can often cache entire datasets in memory. A 100GB cache isn't "large"—it's modest by modern standards.

Storage

Implication: A single database instance can store 50+ TB before you need to consider sharding for storage reasons.

Network

Implication: A 25 Gbps connection transfers ~3 GB/second. Network bandwidth is rarely the bottleneck within a datacenter.

Cost Numbers (2026)

Interviewers increasingly expect cost-awareness, especially for storage-heavy or compute-heavy designs. You don't need exact prices, but knowing the rough cost of a server-hour, a terabyte of storage, and a terabyte of egress lets you reason about trade-offs out loud.

Compute and Storage (AWS on-demand, us-east-1)

Mental model: Storage is cheap, compute is moderate, egress is expensive. A petabyte in S3 Standard is ~$23k/month, which sounds like a lot until you weigh it against the engineering cost of building a custom storage tier to avoid it. The same bytes also span a ~23x price range from hot (Standard) to cold (Deep Archive), so tiering by access pattern is a real lever, not a micro-optimization.

GPU and LLM Inference Numbers (2026)

LLM serving questions now show up in system design interviews, especially at AI companies. Two numbers drive most of the design decisions: KV cache size (which is a memory problem) and prefill cost (which is a compute problem).

GPU Reference

The H200 uses the same compute die as the H100; the only upgrade is a bigger, faster memory subsystem. That alone tells you most inference bottlenecks are about memory, not raw FLOPS.

Model Weights

Weights must fit in GPU memory before you serve a single token. At fp16 (2 bytes per parameter):

KV Cache: the long-context tax

Every token in the context window stores a key and value vector per layer. This KV cache lives in GPU memory alongside the weights and grows linearly with context length.

Formula: KV cache = 2 × layers × kv_heads × head_dim × tokens × bytes

Example (70B-class model: 80 layers, 8 GQA KV heads, head_dim 128, fp16):

Per token: 2 × 80 × 8 × 128 × 2 bytes ≈ 320 KB
1M tokens: 320 KB × 1M ≈ 320 GB

That is ~4 H100s' worth of memory for the KV cache of a single 1M-token request, before counting the ~140 GB of weights. Even an 8B model at 1M context needs ~128 GB of KV cache, which is more than one H100 holds.

Prefill: why time-to-first-token grows

Prefill processes the entire prompt before generating the first token, and it is compute-bound. The dense matmul work is roughly:

Formula: Prefill FLOPs ≈ 2 × parameters × tokens

For a 70B model prefilling 1M tokens, that is ~140 PFLOPs, on the order of tens of seconds on an 8×H100 node at realistic utilization. Attention adds a cost that grows with the square of context length, so at 1M tokens it dominates and pushes real prefill into the minutes.

Component Throughput Limits

These numbers help you understand when to scale specific components.

Databases (PostgreSQL/MySQL)

Caching (Redis/Memcached)

Key insight: A single Redis instance with 500GB of memory can often cache your entire hot dataset. Before adding cache sharding complexity, verify you actually need it.

Application Servers

Key insight: Modern servers have substantial memory. Don't be afraid to use it for local caching or in-memory computations.

Message Queues (Kafka)

Back-of-Envelope Calculations

In interviews, you'll estimate storage, bandwidth, and server requirements. Here's how to approach these calculations systematically.

Useful Approximations

Data Size Reference

Storage Estimation

Formula: Storage = Records × Record Size × Retention × Replication

Example: URL Shortener

Requirements: 100M new URLs/month, 5-year retention

Record size:
- Short code: 7 bytes
- Long URL: 200 bytes average
- Metadata: 50 bytes
- Total: ~260 bytes → round to 300 bytes

Monthly: 100M × 300 bytes = 30 GB
5 years: 30 GB × 60 months = 1.8 TB
With 3x replication: ~5.4 TB

Conclusion: Fits easily on a single database—no sharding needed.

Example: Yelp-like Business Directory

10M businesses × 1 KB each = 10 GB
Add reviews (10x): 100 GB
With indexes and overhead: ~200-300 GB

Conclusion: A single PostgreSQL instance handles this comfortably.

Bandwidth Estimation

Formula: Bandwidth = Requests/second × Average Response Size

Example: Image Service

Requirements: 10M image views/day, average image 500 KB

QPS: 10M / 86,400 ≈ 116 requests/second
Bandwidth: 116 × 500 KB = 58 MB/s = ~500 Mbps
Peak (3x average): ~1.5 Gbps

Conclusion: A CDN with multiple edge nodes handles this easily.

Server Estimation

Formula: Servers = Peak QPS / QPS per Server

Server throughput varies dramatically based on workload:

For back-of-envelope calculations, 10,000 RPS per server is a reasonable default for typical API workloads.

Example: Twitter-like Service

Requirements: 500M DAU, 20 requests/user/day

Daily requests: 500M × 20 = 10B requests
Average QPS: 10B / 86,400 ≈ 116,000 QPS
Peak QPS (3x): ~350,000 QPS

Assuming 10,000 RPS per server (typical API):
Servers needed: 350,000 / 10,000 = 35 servers

Add buffer for redundancy (N+2) and growth: ~50-60 servers for the API tier.

Quick Reference Cheat Sheet

When to Scale Each Component

System Scale Reference

Component Throughput Summary

Common Interview Mistakes

1. Premature Sharding

The most common mistake is proposing sharding for data that fits on a single database.

Bad: "We have 10M businesses at 1 KB each. I'll shard by business_id across 10 nodes."

Good: "10M × 1 KB = 10 GB. Even with 10x for reviews and indexes, we're at 100 GB. A single PostgreSQL instance handles this—no sharding needed."

Same for caches. A LeetCode leaderboard with 100k competitions × 100k users × 40 bytes = 400 GB. This fits on a single large cache instance.

2. Overestimating Latency

SSD random reads are ~100 microseconds, not the 10ms of spinning disks. A simple indexed database lookup takes 1-5ms.

Bad: "Database queries are slow, so we need to cache everything to meet our 100ms latency requirement."

Good: "An indexed lookup takes ~5ms. Our 100ms budget easily accommodates several database calls. We'll cache for throughput, not latency."

3. Unnecessary Message Queues

Adding Kafka for "high write throughput" when a database handles it fine.

Bad: "At 5k writes/second, we need a message queue to buffer the writes."

Good: "5k writes/second is well within PostgreSQL's capabilities (it handles 10-20k TPS). A message queue adds latency and complexity. We'd only add one for guaranteed delivery, async processing, or to decouple producers from consumers."

4. Ignoring Peak vs Average

Bad: "100M requests/day = 1,200 QPS. One server handles that."

Good: "100M requests/day averages 1,200 QPS, but peak might be 5,000+ QPS. We design for peak plus 2x headroom for growth."

5. Using Outdated Numbers

Don't design for 2015 hardware. Modern databases handle terabytes, caches can hold hundreds of gigabytes, and a single server has 256GB+ of RAM. Designing around artificial constraints (sharding at 100GB, worrying about memory limits) signals lack of practical experience.

What Interviewers Look For

When you do capacity estimation in interviews, interviewers evaluate:

1. Grounded estimates — Do your calculations match realistic hardware capabilities?

2. Right-sized solutions — Do you avoid over-engineering while addressing real scale challenges?

3. Clear reasoning — Can you walk through your math and explain your assumptions?

4. Modern knowledge — Are you designing for 2026 hardware, not 2015?

5. Practical trade-offs — Do you understand when to add complexity?

How to Approach Capacity Estimation

Spend 2-3 minutes on estimation early in your design:

1. State assumptions clearly — "Assuming 100M DAU with 10 actions per user per day..."
2. Show your math — Write out the calculation step by step
3. Round generously — Use powers of 10 and round up for safety
4. Draw conclusions — "This means we need X servers" or "A single database handles this"
5. Use results to justify architecture — "Since we only have 100GB of data, we don't need to shard"

This demonstrates the pragmatic thinking that distinguishes senior engineers from those who just memorize solutions.