Interview Framework | ML System Design | DarkInterview
The ML System Design Interview Framework
ML system design interviews are less standardized than classic system design. Different companies emphasize different phases, and every design decision opens three more threads you could chase. Without a structure, strong engineers ramble through features for twenty minutes and never reach evaluation. With one, you cover everything that gets graded and still leave room for depth.
This page gives you that structure: six phases, a time budget for each, what to do in each, and what interviewers are watching for. Treat it as a set of guideposts, not a script. If your interviewer drags you somewhere, follow them; the framework is how you spend the time they leave in your control.
The Framework: 6 Phases
A typical interview runs 45 minutes. Here is how to allocate it:
Phase
Time
Purpose
1. Requirements & ML Objective
~5-7 min
Understand the problem, fix a business objective, translate it to an ML task
2. High-Level Pipeline
~3-5 min
Sketch the end-to-end system so every later discussion has a home
3. Data & Features
~10 min
Decide what the model learns from and how it is represented
4. Modeling
~10 min
Baseline, candidate approaches with trade-offs, one architecture in depth
5. Evaluation & Serving
~8 min
Prove it works offline and online; make it run at production scale
6. Monitoring & Deep Dives
Remaining
Drift, retraining, edge cases, and wherever the interviewer wants depth
Where does extra time go? In 60-minute interviews, put it into Modeling and the Deep Dives: that is where seniority shows.
One hard rule sits under the whole table: no phase gets more than 10 minutes unless the interviewer explicitly pulls you deeper. An incomplete design with one brilliant section scores worse than a complete design with reasonable depth everywhere.
Interviewers evaluate breadth of coverage before depth. The most common failure mode in this interview is not weak ML knowledge; it is spending twenty minutes on the phase you know best and never reaching evaluation or serving. The budget exists to prevent exactly that.
Phase 1: Requirements & ML Objective (~5-7 minutes)
The prompt you get will be deliberately vague: "design a recommendation system," "detect fraud." Your first job is to close the gap between that prompt and a solvable problem. Do it in three moves.
Clarify the problem
Ask targeted questions that change the design, not questions for their own sake:
Who are the users and what surface does this run on?
What scale are we at? Users, items, requests per second?
Does inference need to be real-time or can it run in batches? What latency is acceptable?
What data and labels already exist? Is there a current system we are replacing?
What do mistakes cost in each direction? A blocked legitimate user and a missed fraud case are rarely equally bad.
If the interviewer says "up to you," state an assumption out loud and write it down. Assumptions you name are design inputs; assumptions you hide are landmines.
Fix a business objective
The business objective is what the company actually wants, and it is almost never "maximize model accuracy." A content moderation classifier might exist to reduce harmful exposure, or to reduce legal risk, or to cut human review cost: three different systems. Be specific and directional. "Increase completed purchases from recommendations" gives you a basis for every later trade-off; "improve the experience" gives you nothing.
Pay special attention to where the business objective and the obvious ML objective diverge. If the goal is reducing harmful content exposure, a post about to go viral matters far more than one nobody will see, and that insight should eventually reach your loss function and your evaluation.
Translate to an ML objective
Now name the ML task: classification, ranking, regression, retrieval, generation, or anomaly detection. State what the model predicts, for whom, and roughly how you will know it is good. "A ranking model that predicts purchase probability for each candidate product, evaluated offline with NDCG (a ranking-quality metric) and online with conversion rate" sets up the entire rest of the interview.
Close the phase with thirty seconds of arithmetic: volume per second, and whether the naive "score everything with one model" approach survives the math. It usually does not, and the failed arithmetic derives your architecture (a funnel, a cascade) before you draw anything.
What interviewers look for: questions that reveal you see what makes the problem hard; a business objective specific enough to optimize; an ML formulation that follows from it rather than from habit.
The classic red flag of this phase is naming a model architecture in the first two minutes. Once you say "I'd use a two-tower model," anchoring sets in: every requirement you discover afterward gets bent to fit the choice instead of testing it. Frame first; the architecture will still be there in five minutes.
See Problem Framing for task types, metric alignment, and the estimation math that supports this phase.
Phase 2: High-Level Pipeline (~3-5 minutes)
Before going deep anywhere, sketch the whole system so the interviewer can see where each later discussion fits. For most problems the skeleton looks like this:
Walk the full lifecycle out loud: where data comes from, how the model trains, what happens on a request, what action the prediction triggers, and how outcomes flow back as training signal. That dotted feedback edge is worth drawing every time; half of the interesting production problems (drift, feedback loops, cold start) live on it.
Two cautions. First, the diagram is a communication aid, not a deliverable: do not polish boxes while the clock runs. Second, stay out of SWE-level detail like database choices and API schemas. In applied ML interviews the infra is assumed; the signal is in the ML decisions.
Update the diagram as the design evolves. When you add a cascade in the serving discussion or a re-scoring path in data, spend the five seconds to draw the box. A diagram that still matches your final design is a free demonstration of coherence; a stale one actively misleads.
What interviewers look for: a complete lifecycle, including the feedback path, delivered quickly. Red flags are perfectionism on the diagram and premature infrastructure depth.
Phase 3: Data & Features (~10 minutes)
This is the phase where candidates most often sink. There is effectively infinite material to discuss, so the skill being graded is prioritization: name the categories, pick the high-signal items in each, and keep moving.
Training data
Cover three buckets, in order of how often candidates forget them:
Labeled data. What exists, how much, how good, and how you get more. Discuss the labeling process and its cost.
Weak and proxy labels. User actions like clicks, reports, purchases, and appeals are noisy but arrive at orders of magnitude more volume than curated labels.
Unlabeled data. Raw content and behavior logs that can power pretraining or self-supervised objectives. Great answers usually exploit this bucket; average answers never mention it.
Name the data problems that apply: class imbalance, label delay, selection bias, cold start. You do not need to solve them all now; flag them and promise to return. Details in Training Data & Labels.
Features
Organize by signal source instead of free-associating: the item being scored, the user or actor involved, behavioral aggregates, network or graph structure, and request context. Propose two or three high-value features per relevant source with a hypothesis for why each is predictive, then say how each is represented: raw text or images through an encoder, categorical IDs through embedding tables, counters as normalized or smoothed numerics. Note which features are precomputed and which must be fresh at request time; that split drives serving cost later.
What interviewers look for: creative use of non-labeled data; features with stated hypotheses rather than a laundry list; representation and freshness handled, not hand-waved.
The feature dump is the classic Phase 3 failure: thirty features, no priorities, ten minutes gone, and the interviewer has learned nothing except that you cannot prioritize. Two or three features per source, each with a why, then move. Earmark the rest.
Propose the simplest thing that could work: a popularity heuristic, logistic regression on your top features, a gradient-boosted tree. This is not filler. It grounds every later choice in a comparison ("what does this complexity buy over the baseline?"), and in multi-component systems a baseline can stand in for one stage while you spend your depth on another.
Compare candidate approaches
Lay out two or three real options spanning classical and deep learning, and compare them on the axes that matter for this problem: quality ceiling, latency, training data requirements, interpretability, and operational cost. Deep learning is not automatically the answer; saying so, with reasons, is a senior signal. Then commit to one. You do not have time to elaborate them all.
Go one level deep
For your chosen approach, describe the architecture at the level of someone who has built one: the main components, how inputs enter, the loss function, and how you would fight overfitting. Aim between "I'd use a transformer" (too shallow) and inventing layer sizes you could only know empirically (fake precision). Expect follow-up questions here; this is where interviewers test whether you have really trained models or only read about them.
It is fine to admit unfamiliarity with a domain: "I've built classification systems, not ranking models, but here is how I'd adapt what I know." The interview's central question is whether your knowledge generalizes to a new space. Showing the adaptation reasoning openly is how you answer it; bluffing depth you lack is how interviewers find the bottom of your knowledge on the next follow-up.
What interviewers look for: baseline first; trade-offs argued, not recited; architecture detail that survives probing. Red flags are reaching for the most complex model immediately and hand-waving under follow-up.
Build the metric story in layers that connect back to Phase 1: the business objective, the product metrics that reflect it, and the ML metrics you can compute cheaply. Then describe the methodology: offline evaluation on a held-out set (say which metrics and why they fit the task), and online evaluation through an A/B test measuring the business metric. Name the gap explicitly: offline gains do not always survive contact with users, so you validate that your offline metric actually predicts online wins, and you investigate when they diverge.
If the problem has an evaluation trap, this is where you flag it: presentation bias in click data, delayed labels in fraud, subjectivity in generative output.
Serving
Now make it run. Cover the serving pattern (real-time vs. batch), the latency and cost math at the stated scale, and the standard toolkit when the numbers do not fit: cascade architectures that run a cheap model on everything and an expensive model on the survivors, distillation, quantization, and caching of expensive encodings. Say when the model runs: on every request, on content creation, re-triggered by new signals?
What interviewers look for: metrics tied to the business objective rather than free-floating AUC; awareness of the offline-online gap; serving numbers that acknowledge the scale from Phase 1.
The notebook mindset is the Phase 5 red flag: a model with beautiful offline metrics and no path to production. If your design never mentions latency, cost, or what happens when the model errors, an applied-role interviewer will conclude the gap themselves.
Close the loop, then go wherever the remaining minutes are most valuable.
Monitoring is the minimum. Say what you track in production (prediction distributions, feature distributions, online metrics), what triggers an alert, and how retraining happens. Distinguish data drift from concept drift and note the feedback loop: once your model acts on the world, it shapes its own future training data. Vague "we'd retrain periodically" answers leave points on the table; scheduled retraining plus drift-triggered retraining plus a holdout to measure honestly is a complete answer. See Monitoring & Drift.
Deep dives are the differentiator. Offer a shortlist from the threads you earmarked earlier: cold start, explore/exploit, feedback loops, adversarial behavior, fairness, scaling training. Let the interviewer pick, and if they drive somewhere specific, follow at full speed. A strong closing move: "With the time left I could go into cold start, the feedback loop, or inference cost. Any preference?"
What interviewers look for: production maturity in the monitoring story, and genuine depth in at least one dive. The red flag is steering to a rehearsed pet topic while ignoring the interviewer's signals.
A Complete Opening, As It Sounds
The first two phases, spoken, for a moderation prompt:
"Before designing anything I want to pin the problem down. Posts are text plus images? What volume, and how rare is harmful content? What actions can we take, and what does a wrong removal cost us? ... OK: 500M posts a day, under 1% harmful, auto-remove needs 95% precision, small review team. The objective I'd propose is minimizing views of harmful content subject to that precision bar, because exposure is the harm and it makes latency questions answerable. That means a calibrated multi-modal classifier with thresholds for remove, demote, and review. Volume math is about 6K posts a second, so I already expect a cheap filter in front of an expensive model. Here is the pipeline..."
Ninety seconds, and the interviewer has scope, objective, formulation, arithmetic, and a preview of the architecture. Every worked question in this track (video recommendations, harmful content, search ranking, fraud) opens exactly this way.
When to Break the Order
The order above is the default, not a law. Two situations justify rearranging it, and doing so deliberately reads as seniority:
Constraints dominate the design. For "ad click prediction at a million QPS," the latency and cost budget eliminates whole model families before you would normally discuss them. Pull serving constraints forward into Phase 2 and let them shape modeling.
Evaluation is the hard part. For generative systems (a RAG assistant, a summarizer) there is no single right answer, so agreeing on what "good" means cannot wait until Phase 5. Discuss evaluation right after the objective.
The rule is to announce the deviation: "Serving constraints will drive everything here, so I want to cover them before picking a model. Sound good?" A silent reordering looks like disorganization; a named one looks like judgment.
Managing the Clock
Three habits keep the framework on schedule:
Signal transitions. One sentence between phases ("That is the data story; moving to modeling unless you want more here") gives the interviewer a natural place to redirect you and proves you are structured.
Earmark instead of diving. When a rabbit hole opens mid-phase, name it and defer it: "There is a feedback loop risk here; I will come back to it in deep dives." You get credit for spotting it without paying the time. If the interviewer wants it now, they will say so.
Recover without ceremony. When you realize you have overrun a phase, summarize your position in one sentence, state your decision, and move. No repeated apologies. Catching yourself is a positive signal; dwelling on it is not.
Three warning signs that you are off the rails, and the recovery for each:
Warning sign
What it means
Recovery
3+ minutes on one component with no pipeline reference
Rabbit hole
"Let me zoom out and place this in the pipeline; we can return if useful."
No business metric mentioned in 5 minutes
Technical drift
"Tying this back: the reason it matters is [objective]."
No number mentioned in 10 minutes
Hand-waving altitude
State a scale, a latency, or a size; re-anchor in constraints.
Interviewer redirects are gifts, not interruptions. "How would you evaluate this?" mid-modeling means they have seen enough modeling; the highest-scoring response is a clean transition, not a defense of the unfinished thought.
Remember the goal. Nobody designs a production ML system in 45 minutes. You are demonstrating a thought process: how you interrogate a problem, make trade-offs under constraints, and connect technical decisions to business outcomes. The framework exists so that the demonstration is complete.