Machine Learning Engineer interview question
How do you roll back a model release?
Keep model, preprocessing, feature schema and configuration as one immutable release; route traffic through a version switch, retain the prior artifact, define health and quality rollback signals, and avoid incompatible feature changes.
Direct answer
What a strong answer covers
How it works
A model release is only reversible if it is one unit: the model weights, the preprocessing code, the feature schema version, and the runtime configuration are versioned and deployed together, because rolling back the weights while keeping a new feature schema produces a model that runs and answers wrongly. Traffic goes through a router that can shift percentages between versions in seconds, the last healthy release stays warm or is quick to restore, and rollback is triggered automatically on service signals such as error rate and latency. Quality signals that need delayed labels get a separate, human-decided rollback path with its own thresholds.
Example
Version 22 of a pricing model was promoted to 20 percent of traffic behind a router; error rate stayed flat but a 2 percent drop in conversion appeared within six hours in the canary cohort, below the agreed guardrail, and the on-call engineer moved traffic back to version 21 with one config change. Because version 22 had introduced a new feature, the feature pipeline kept computing both versions for a week so the rollback did not break serving.
Common mistakes
- Rolling back model weights while leaving a new feature transformation in place.
- Deleting the previous model artifact after promotion.
- Relying only on service health and missing a quality regression that needs business metrics.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.