Start with a decision
Describe the user problem, proposed change, expected outcome and reasoning. Choose a primary metric before launch. Decide which outcomes would make you pause the test, how long you will measure and who will review the decision.
Allocate traffic
Traffic allocation determines what share of eligible visitors enters the experiment. Variant weights determine the split within that traffic. Allocation uses basis points: 10,000 equals 100%. All variant weights together must total 10,000.
A stable visitor identifier, experiment identity, domain and allocation version determine assignment. Keep identifiers consistent across requests and integrations. Do not change the assignment protocol or identifier during a running experiment.
Plan the horizon
Record the minimum detectable effect and a planned sample size or duration before publishing. The built-in two-arm conversion planning estimate uses a 95% confidence, 80% power normal approximation. Revenue and multi-variant plans need a design appropriate to their variance and comparisons.
Repeatedly checking a p-value and stopping when it first becomes small is not a valid fixed-horizon stopping plan.
Drafts and started experiments
Drafts can be edited. After launch, allocation, targeting and the analysis plan are frozen. To test a different hypothesis, audience or allocation plan, create a new experiment. This preserves the meaning of recorded results.