Data Engineer interview question
How do you implement data lineage?
Capture lineage from source fields through transformations to datasets and dashboards using versioned metadata emitted by jobs, then connect it to ownership, schema changes, quality incidents, and impact analysis.
Direct answer
What a strong answer covers
How it works
Lineage records where each dataset and column came from and what depends on it. It is captured automatically where possible: transformation tools such as dbt emit model dependencies, orchestrators emit task inputs and outputs through OpenLineage, and query logs reveal which tables feed which dashboards. The graph is stored with versions and timestamps in a catalogue, tied to ownership and schema metadata, so an engineer can ask what breaks if a column is dropped and an analyst can ask where a number on a dashboard came from. Its value shows up during incidents and migrations, which is when hand-maintained diagrams are always out of date.
Example
When a source system deprecated a field, the catalogue showed it fed 3 dbt models, 2 of which powered the board-level revenue dashboard, so the migration was scheduled before the field disappeared instead of after the dashboard broke. During a quality incident, the lineage graph pointed from the wrong figure back to a transformation changed the day before.
Common mistakes
- Maintaining lineage in a wiki that is stale within a month.
- Capturing table-level lineage only when the questions are about columns.
- Collecting lineage nobody can query because it is not linked to owners or dashboards.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.