How to Read the Evidence

Key takeaways

• Key figures in this book carry grades A–D. A means randomized experiments; B, observational studies and large primary datasets; C, vendor aggregates and self-reported surveys; D, individual cases and secondary citations.
• Conversion rates with different denominators are different metrics. Mixing sessions, users, messages sent, and recipients can produce differences of more than 10 times.
• Vendor benchmarks sample the vendor's customers. They include companies mature enough to measure performance, so new teams should base decisions on outcomes near the lower end of the range.
• Dividing average CPC by average CVR does not produce average CPL. These metrics are averaged separately; treat them as ranges.
• Do not put the double-digit lifts from case studies straight into a plan. An estimate based on experiments at Qubit customers placed 90% of treatment effects within ±1.2% (the authors describe this as a rough indicator).

Evidence Grades

Methodology references (textbooks, model papers, and guidelines) do not provide numerical evidence, so they are either ungraded or assigned B. Meta and Google research with platform-affiliated coauthors is identified in the text.

Align the Denominator First

Check four things before comparing numbers.

Attribution and Incrementality Differ

Most vendor CVR, CPL, and ROAS figures are attributed results. They include people who would have bought without seeing the ad. In the eBay experiment, switching off nonbrand search ads reduced attributed revenue by more than 72%, but total revenue showed no clear change. See Reported and Incremental Conversions for adjustment details.

Six Rules for Using Benchmarks

Terms Used in This Book

Numerical Conventions

• Grades appear in a table's Grade column or at the end of its evidence cell; Korean case tables put them at the end of the first cell. Prose uses (grade C).
• Calculated means this book divided, multiplied, or converted source values; the result does not appear verbatim in the source.
• Secondary citation means the original publication and underlying data were not both reviewed; the value was confirmed only through another article. It is grade D.
• Via press coverage means public data or vendor aggregates were reviewed only through reporting. The grade follows the nature of the underlying data.
• Summary means only an abstract or institutional summary was reviewed.
• Author judgment marks judgments, interpretations, or estimates that do not follow directly from the evidence.
• Dollar amounts are preserved as published because there is no basis for converting them into Korean-market benchmarks.
• Measurement periods are included. Do not mix figures from different editions.

Next Steps

• Find relevant figures in the Conversion Atlas.
• Turn them into distributions in Conversion Rate Intervals.