Data Scientist interview question
What is the bias-variance trade-off?
High bias underfits because the model is too constrained; high variance overfits because it follows training noise. Balance them with appropriate capacity, regularization, representative data, and validation on unseen samples.
Direct answer
What a strong answer covers
How it works
Expected prediction error decomposes into bias, the systematic gap between what the model can represent and the true relationship, variance, the sensitivity of the fitted model to the particular training sample, and irreducible noise. A model with too little capacity has high bias and misses signal in both training and validation; a model with too much capacity relative to the data memorises noise, so training error is low and validation error is high. The lever is the balance between capacity and data: more or cleaner data, regularisation, ensembling, and feature choices, all judged on held-out data rather than training fit.
Example
A linear model predicting house prices from square footage underfits, with a similar 20 percent error on training and validation. A gradient-boosted model with a thousand deep trees reaches 2 percent training error and 25 percent on validation, the signature of variance. Limiting depth, adding L2 regularisation, and early stopping on a validation fold brought validation error to 11 percent, and the learning curve showed more data would still help.
Common mistakes
- Reading a low training error as a good model.
- Adding capacity to fix underfitting without checking that the data can support it.
- Tuning hyperparameters on the same fold used to report the final score.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.