Data Scientist interview question
What is data leakage in machine learning?
Data leakage occurs when training features or preprocessing contain information unavailable at prediction time or derived from the target. It produces optimistic validation and fails in production; split before fitting every learned transform.
Direct answer
What a strong answer covers
How it works
Leakage is any way information about the target reaches the model that will not exist at prediction time. Target leakage is a feature computed after or from the outcome, such as a refund flag when predicting fraud. Temporal leakage is training on rows from after the prediction date or aggregates that include future events. Preprocessing leakage is fitting scalers, encoders, or imputers on the full dataset before splitting. Entity leakage is the same user appearing in train and test. The symptom is validation performance that looks too good; the cure is building features as of the prediction timestamp and fitting every transformation inside the training fold.
Example
A model predicting loan default reached 0.98 AUC because one feature was the number of collection calls, which only happen after default. Removing it dropped AUC to 0.79, which matched production. A second issue appeared when a customer-level average was computed over all history including the label month; recomputing it as of the application date fixed the remaining optimism.
Common mistakes
- Treating a suspiciously high score as good news instead of a warning.
- Computing aggregate features over the whole dataset including the future.
- Fitting an imputer on all rows and then splitting.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.