Product Manager interview question
How do you respond when a product experiment fails?
Treat a failed experiment as evidence: verify instrumentation and execution, quantify uncertainty, compare the result with the original hypothesis, inspect segments without fishing, decide stop, iterate or retest, and document the learning.
Direct answer
What a strong answer covers
How it works
A failed experiment is information, and the first job is to make sure it is accurate information. Check that the variant actually shipped to the intended users, that events were tracked on both arms, and that the sample and duration matched the plan. Then quantify: the effect with its interval, not just the sign, and whether the result rules out the hypothesised effect or merely failed to confirm it. Compare against the original hypothesis in writing, look at pre-registered segments without fishing for a winner, and decide explicitly to stop, iterate on a specific learning, or rerun with a corrected design. The write-up is shared so the organisation does not repeat the same bet.
Example
A redesigned pricing page was expected to lift trial starts by 5 percent and showed minus 1 percent with an interval from minus 4 to plus 2. Instrumentation was verified, the sample was as planned, and the pre-registered segment of visitors from paid ads showed the same null. The team concluded the layout was not the barrier, and follow-up interviews pointed at unclear plan differences, which became the next experiment. The result was posted in the experiments log with the reasoning.
Common mistakes
- Digging through segments until one shows a positive result.
- Declaring the idea dead without checking whether the test was actually valid.
- Hiding the failure so the same experiment is proposed again next quarter.
A useful answer structure
Answer from your own work.
Do not memorise a script: use only examples you can explain honestly in follow-up questions.
Open the desktop workspace and rehearse a clear answer in your own words.