Data Engineer interview questions and answers.

Questions

Technical questions to practise out loud

1. 01

   What is the difference between ETL and ELT?

   ETL transforms data before loading it into the target; ELT loads raw data first and transforms inside a capable warehouse. Choose from governance, latency, compute location, reproducibility, and access-control needs.

   Read the full breakdown
2. 02

   How do you choose a data partitioning strategy?

   Partition by columns that match common filters and distribute volume reasonably, while avoiding tiny partitions and skew. Plan retention, compaction, late data, metadata cost, and how queries prune partitions.

   Read the full breakdown
3. 03

   How do you approach exactly-once data pipelines?

   Exactly-once business effects require replayable input, deterministic processing, idempotent or transactional writes, stable event identities, and checkpointing. A broker's delivery guarantee alone does not make the whole pipeline exactly once.

   Read the full breakdown
4. 04

   How do you choose a data model?

   Choose a data model from consumers, grain, update patterns, history, relationships, query latency, and governance. Preserve one clear source of truth, then optimize warehouse facts, dimensions, or denormalized views for access.

   Read the full breakdown
5. 05

   How do you build data quality checks?

   Run checks at ingestion and transformation boundaries for schema, nulls, uniqueness, referential integrity, ranges, freshness, volume, and distributions; quarantine bad data and assign ownership and severity.

   Read the full breakdown
6. 06

   How do you handle schema evolution?

   Evolve schemas compatibly: add optional fields first, version contracts, let old and new producers coexist, validate consumers, backfill when required, and remove fields only after usage and retention windows expire.

   Read the full breakdown
7. 07

   When do you choose streaming over batch?

   Choose streaming when the decision value requires low latency and continuous state; choose batch when bounded delay simplifies correctness and cost. Include event-time ordering, late data, replay, and operational burden.

   Read the full breakdown
8. 08

   How do you orchestrate data pipelines?

   Model pipelines as idempotent tasks with explicit dependencies, inputs, outputs, retries, timeouts, backfills, concurrency limits, and ownership; persist run state and expose data-quality and freshness signals.

   Read the full breakdown
9. 09

   How do you implement data lineage?

   Capture lineage from source fields through transformations to datasets and dashboards using versioned metadata emitted by jobs, then connect it to ownership, schema changes, quality incidents, and impact analysis.

   Read the full breakdown
10. 10

    How do you control data warehouse costs?

    Control warehouse cost by attributing spend to teams and workloads, pruning partitions, selecting only needed columns, reducing repeated scans, materializing reused results, sizing compute, scheduling work, and setting budgets and alerts.

    Read the full breakdown