Data Engineer interview question
When do you choose streaming over batch?
Choose streaming when the decision value requires low latency and continuous state; choose batch when bounded delay simplifies correctness and cost. Include event-time ordering, late data, replay, and operational burden.
Direct answer
What a strong answer covers
How it works
Streaming pays off when a decision loses value within seconds or minutes, such as fraud checks, live inventory, or operational alerts, or when continuous state such as a session must be maintained as events arrive. Batch is simpler and cheaper when a delay of an hour or a day changes nothing, because bounded inputs make correctness, reruns, and testing easier. Streaming brings event-time versus processing-time, watermarks for late data, exactly-once concerns, and a system that must be operated around the clock, so the decision is made per use case with the required latency written down rather than assumed.
Example
A marketplace scores each new listing for fraud within five seconds using a Flink job keyed by seller with a 24-hour window, because a fraudulent listing that stays visible for an hour costs real money. The same company computes seller payouts nightly in batch, since payouts are settled daily anyway and a rerun after a bug fix is trivial with bounded input.
Common mistakes
- Streaming everything because it sounds modern and paying for 24/7 operations on daily reports.
- Ignoring late events and producing windows that are wrong for the last hour of each day.
- Building batch for a use case whose value expires before the batch runs.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.