Reading experiment readouts

When an experiment concludes, its readout tells you what happened and whether to trust it.

What the readout covers

• The effect — how the primary metric moved in the variant versus the control.
• Reliability — whether the result is strong enough to act on, or whether the test needs more time or data.
• Guardrails — whether any secondary metric was harmed in the process.

How to read it

Do not just look at the headline number — look at whether the result is reliable. A small, noisy difference is not a win. A clear, robust effect that did not harm your guardrails is a decision you can make confidently.

From readout to rollout

If the variant won reliably, roll it out — ideally via Write-back — and record the learning. If it lost or was inconclusive, that is still valuable: you avoided a change that would not have helped.