Experiment Design
Choose the right MVP experiment and define success criteria before building.
Key takeaways
- An experiment is the smallest product-shaped test that yields credible evidence, sized to the decision rather than the team's excitement.
- The experiment brief fixes seven fields up front: hypothesis, segment, offer, test surface, success metric, guardrail, and decision rule.
- Pick one primary metric plus a few diagnostics, prefer real behavior over stated preference, and make failure interpretable.
- Set concrete success criteria before building, such as 8%+ visitor-to-demo conversion or 3 paid pilots.
An experiment is the smallest product-shaped test that can produce credible evidence. The build should be sized to the decision, not to the team's excitement.
Experiment Brief
| Field | Description |
|---|---|
| Hypothesis | The one claim being tested |
| Segment | Who the test targets |
| Offer | What the user sees or receives |
| Test surface | Landing page, prototype, concierge, paid pilot |
| Success metric | Observable behavior that supports the hypothesis |
| Guardrail | Safety, brand, legal, or support constraint |
| Decision rule | What happens after the result |
Design Rules
- Pick one primary metric and a few diagnostic metrics.
- Prefer real behavior over stated preference.
- Keep the path short enough to instrument completely.
- Make failure interpretable.
- Decide what not to build.
Sample Success Criteria
| Goal | Metric |
|---|---|
| Validate problem urgency | 10 qualified interviews with repeated pain language |
| Validate offer | 8%+ qualified visitor to demo conversion |
| Validate willingness to pay | 3 paid pilots or written budget confirmation |
| Validate activation | 60%+ of pilot users complete the core action |
Experiment Shape
A good experiment has a narrow customer, a clear promise, a controlled path, and a decision rule.