MLOps Engineer interview question
How do you control MLOps platform costs?
Attribute compute, storage and serving cost by team, experiment and model; enforce quotas and idle shutdown, reuse cached features, right-size accelerators, schedule training, use spot safely, and optimize cost per business outcome.
Direct answer
What a strong answer covers
How it works
Platform cost is managed by making it attributable and then bounded. Every job, endpoint, and storage bucket carries tags for team, project, and model so spend can be reported against outcomes. Quotas per team cap GPU hours and endpoint counts, idle shutdown reclaims notebooks and dev endpoints, and spot capacity with checkpointing serves restartable training. Feature computation is cached and shared rather than recomputed per experiment, accelerators are sized from profiling, training is scheduled into cheaper windows, and inference is batched where latency allows. The review metric is cost per business outcome, such as cost per served prediction or per approved model, trended monthly.
Example
A platform team publishing a monthly cost report per model found that a single experiment sweep consumed 30 percent of GPU hours with no promoted model; a quota per team with an override process and spot instances for sweeps cut that line by 70 percent. Idle-notebook shutdown at 45 minutes reclaimed another 12 percent, and the report now shows cost per promoted model alongside raw spend.
Common mistakes
- Reporting total platform spend with no attribution so nobody owns a line.
- Running restartable training on on-demand instances.
- Provisioning a GPU endpoint for a model that could be scored in a nightly batch.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.