Problem Framing | ML System Design | DarkInterview
Problem Framing
Most failed ML design interviews are lost in the first five minutes, before any model is named. The prompt is intentionally ambiguous, and the interviewer is watching whether you convert it into a precise, solvable problem or anchor on the first architecture that pattern-matches. This page covers the tools for that conversion: the clarifying questions worth asking, the objective hierarchy, the catalog of ML task types, and the estimation math that grounds your choices in reality.
Why Framing Decides the Interview
"Design a recommendation system" could mean home feed ranking, related-item suggestions, or push notification selection. Each has different objectives, latency budgets, data, and evaluation. Committing to a design before pinning this down means designing the wrong system with confidence.
Consider how differently the same prompt resolves:
Prompt
One valid reading
Another valid reading
What changes
"Design a recommendation system"
Home feed ranking
Email digest personalization
Real-time vs. batch, latency by 100x
"Design search for Airbnb"
Guest search, optimize bookings
Host-side competitive research
Objective, users, features, everything
"Detect fraud"
Card fraud at checkout, 100ms
Merchant fraud, offline review
Latency, actions, label sources
"Recommend videos"
Maximize watch time
Reduce churn
Loss function, evaluation, value model
There is also a subtler trap: anchoring. The moment you say "I'd use a two-tower model," every subsequent thought bends toward justifying it. Requirements you discover later get squeezed into the choice instead of reshaping it. Framing first keeps the solution space open until the constraints have spoken.
Framing buys you a decision audit trail. When the interviewer later asks "why two stages?", the answer is "because we established 200ms latency over a billion-item catalog," not "it's a common pattern." One of those answers demonstrates engineering; the other demonstrates memorization.
Five categories cover what you need. Two or three questions per relevant category is plenty; the goal is design-changing information, not an interrogation.
Category
Ask about
Why it changes the design
Business objective
What outcome matters: revenue, engagement, safety, cost? What does success look like?
Shapes the loss function, metrics, and every trade-off
Scale
Users, items, requests per second, geographic spread
Decides single-model vs. multi-stage architectures
Data
What labels exist, their quality and delay, fraction of new users/items
Determines feasible model families and cold-start priority
Latency / cost
Real-time or batch? Latency budget? Compute constraints?
Eliminates whole architecture classes early
Consequences
Cost of false positives vs. false negatives, review capacity, appeal process
Sets thresholds, calibration needs, and human-in-the-loop design
A few sample questions per category, phrased the way you would actually say them:
"Who consumes the output, and on what surface? A ranked page, an automated action, a review queue?"
"What scale should I design for: how many users, items, and requests per second?"
"Do we have labeled data today? How much, how good, and does it keep arriving?"
"Is inference in the request path? What latency can we afford?"
"Which error hurts more here: a false positive or a false negative? By how much?"
If the interviewer answers "up to you," make the assumption explicit and visible: "I'll assume 100M DAU and real-time serving; correct me if that changes the problem." Senior candidates ask fewer questions and state more assumptions; the skill is the same either way.
Skip questions with no design impact. "What language is the backend in?" tells the interviewer you are stalling. Every question you ask should be one whose answer would change what you build. And know when to stop: five to eight minutes of scoping is the sweet spot; beyond that you are starving the design phases.
From Business Objective to ML Objective
Work down a three-level hierarchy, and keep the levels distinct:
Business objective: the outcome the company wants. "Reduce the number of times users see harmful posts." "Increase completed bookings."
ML objective: the prediction task that serves it. "Classify posts as harmful with calibrated probabilities." "Rank listings by booking likelihood."
Loss function: the quantity the optimizer actually minimizes. Chosen last, and often adjusted to reflect the business objective (for example, weighting examples by expected exposure).
Where the levels diverge is where the design lives
The interesting design work happens where these levels disagree, and interviewers deliberately pick problems where they do:
Harmful content: a classifier that optimizes plain accuracy treats a post seen by ten million people the same as a post seen by nobody. The business objective (reduce exposure) says they are enormously different. Resolution: exposure-weighted loss and exposure-weighted evaluation.
Recommendations: click prediction optimizes something users do impulsively; the business wants retention. Resolution: multi-task heads plus a value model that blends predictions with tuned weights.
Fraud: classification accuracy ignores that a $10,000 fraud and a $10 fraud are different events. Resolution: dollar-weighted loss and a decision layer with value-aware thresholds.
Getting one of these divergences on the table early, unprompted, is one of the strongest signals you can send in the first ten minutes.
Habits that keep the hierarchy honest
Be directional, not fake-precise. "Increase click-through on recommendations" is actionable. "+10% revenue" is theater when nobody stated current revenue.
Ask what action follows the prediction. A fraud score that triggers an automatic block, a manual review, or a step-up authentication leads to three different systems. The action determines the cost of errors, and the cost of errors determines the objective.
Name the guardrails alongside the objective. "Maximize X" is rarely the full statement; it is "maximize X subject to Y not degrading." Precision floors, false-positive caps, and latency budgets are part of the objective, not afterthoughts.
Teams at large companies spend years optimizing one narrow business objective, and your interviewer probably works on one. The bar is not inventing the perfect objective in two minutes; it is showing you understand that the objective is a designed artifact with consequences, not a given.
Choosing the ML Task Type
With the objective fixed, name the task. Seven types cover nearly every interview prompt:
Task type
Output
Recognize it by
Typical examples
Typical loss
Classification
A discrete label (with probability)
"detect", "flag", "categorize"
Spam, fraud, content moderation
Cross-entropy
Ranking
An ordered list
"show the most relevant", "top K"
Search results, feeds, ads
Pairwise/listwise ranking losses
Retrieval
A small candidate set from a huge corpus
Billions of items, tight latency
Recommendation candidates, RAG
Contrastive (InfoNCE, triplet)
Regression
A continuous number
"predict", "estimate", "forecast"
ETA, demand, bid pricing
MSE / MAE / quantile
Generation
New content
"generate", "summarize", "answer"
Assistants, code completion
Cross-entropy over tokens
Anomaly detection
A rarity/deviation flag
Extreme imbalance, novel attacks
Fraud, abuse, infra monitoring
Reconstruction / one-class
Clustering
Groups without labels
"segment", "discover"
User segmentation, topic discovery
Distance-based objectives
Three things make this more than a vocabulary quiz:
Many problems admit multiple framings
Airbnb search could be classification (book or not per listing), ranking (order by booking likelihood), or regression (expected booking value). The choice is a design decision with a defensible process:
What does the consumer of the output need? A results page needs an ordering; an automated gate needs a calibrated probability; a bid needs a number.
Which framing best serves the business metric? Revenue argues for value regression; conversion argues for ranking by booking probability.
What do serving constraints allow? Ranking implies scoring a candidate set under latency; independent classification is cheaper but loses slate-level ordering.
Name the framing you rejected and why. Silent framing choices read as habit; argued ones read as judgment.
Real systems compose task types
A production recommender is retrieval followed by ranking. A moderation pipeline is classification with an anomaly-detection side channel for novel abuse. Search is retrieval, ranking, and a bit of classification (intent) stacked. Saying "this decomposes into a retrieval problem and a ranking problem" early makes the rest of the interview dramatically easier to structure, because each sub-problem gets its own model, metrics, and serving story.
Ask the zeroth question: does this need ML at all?
If a rules-based system or a simple heuristic clears the bar, saying so is a credibility win, not a dodge. The strongest version proposes the heuristic as the v1 and the ML system for the residual: "static price thresholds catch the blatant fraud; the model earns its keep on the ambiguous middle." Interviewers who run production systems know that ML is a maintenance liability, and candidates who reach for it reflexively read as junior.
Task-type choice constrains serving before you ever discuss serving. A generation task means autoregressive decoding and hundreds of milliseconds; a classification task is a single forward pass in single-digit milliseconds. When you name the task type, you are implicitly signing up for its latency profile, and interviewers notice when the signature bounces.
Back-of-the-Envelope Math for ML Systems
A few quick calculations, done at framing time, prevent both over- and under-engineering. Interviewers grade the reasoning chain and the stated assumptions, not third-decimal precision. The generic estimation toolkit in Numbers to Know still applies; these are the ML-specific additions.
Model memory
Parameters times bytes per parameter:
Precision
Bytes/param
7B model
Typical use
FP32
4
28 GB
Legacy training
FP16 / BF16
2
14 GB
Standard inference and mixed-precision training
INT8
1
7 GB
Quantized serving
Training takes several times the memory of inference. Adam-style optimizers keep gradients and momentum alongside weights, so a working rule is 12-16 bytes per parameter before activation memory, versus 2-4 bytes for inference: roughly 4-8x depending on the precision mix. Quoting inference memory for a training question is a classic miss:
7B model, inference (FP16): 7B x 2 bytes = 14 GB (one large GPU)
7B model, training (Adam): 7B x ~14 bytes = ~98 GB (multi-GPU territory)
Embedding table storage
Entities times dimension times bytes:
100M users x 128 dims x 4 bytes = ~51 GB (fits in memory, shardable)
1B users x 64 dims x 4 bytes = ~256 GB (this is why the hashing trick exists)
For billion-scale ID spaces, hash IDs into a fixed bucket count (say 10M) and let rare IDs share rows: heavy users dominate their bucket's gradient and stay sharp, long-tail users blend toward an average that was all their data supported anyway. Multiply by ~3x replication when sizing real storage.
Serving throughput
Work demand against supply:
Demand: peak QPS x candidates scored per request
Supply: instances x (batch size / latency per batch)
Example: 5K QPS x 500 candidates = 2.5M scores/sec
At 10K scores/sec/GPU -> ~250 GPUs for the heavy ranker alone
-> this is why the funnel narrows to ~500 before the heavy model
Two caveats worth voicing: batching raises throughput but adds latency, so the right batch size depends on which constraint binds; and real GPU utilization runs 30-50% of the datasheet number once data loading and synchronization overheads land.
A worked micro-example
"Classify 500M posts per day" sounds enormous until you divide: about 6K posts per second average, maybe 15K at peak. If a distilled first-stage model clears one post per millisecond per core, a few dozen cores cover the easy 95%, and the expensive model sees only the residual. Thirty seconds of arithmetic just justified a cascade architecture and a hardware budget.
Round aggressively and sanity-check against anchors: a large GPU has tens of GB of memory; a modern CPU core does simple model inference in fractions of a millisecond; 1B/day is ~12K/sec. Fluency with three or four anchors beats memorizing tables.
Putting It Together
A complete framing, spoken aloud. The Interview Framework shows this move on a moderation problem; here is the same technique on fraud, so you can see the pattern is portable:
"So we are building stolen-card fraud detection at checkout: roughly 10K transactions a second at peak, well under half a percent fraudulent, and the decision must land inside a 100ms authorization window. The business objective I would propose is minimizing total dollar losses, fraud plus falsely declined good sales, subject to a per-segment false-decline guardrail since repeat false declines lose customers. That makes the ML objective calibrated fraud-probability scoring feeding a three-way decision: approve, step-up verification, or decline. The costs are dollar-denominated per transaction, which should reach the loss weighting and the thresholds. One wrinkle to flag now: chargebacks confirm fraud 30 to 90 days late, so training and evaluation will have to respect label maturity. Let me sketch the pipeline."
Scope, objective with guardrails, formulation, cost asymmetry, and the domain's defining data problem, in under a minute. Everything after this gets easier, because every later decision has something to be checked against.
Quick Reference
Framing checklist
Step
Output
Clarify
Users, scale, latency, data, error costs on the whiteboard