Evaluation

An ML engineer's real job is running an optimization loop, and evaluation is the loop's instrument panel. Interviewers weight this phase heavily because it separates candidates who have shipped models from those who have trained them in notebooks: a model that wins on offline metrics but moves no business number is a science project. This page covers the metric stack, offline metrics by task type, online experimentation, the divergence between the two, guardrails, and the evaluation traps specific to each system type.