120,000 reachable households, scored by response to advertising
Most of the households your propensity model loves were going to buy anyway. One group buys less when you advertise to them — and conventional targeting reaches that group as soon as the budget is large enough.
| Households reached | Spend | Who will buy | Who we can change | Marginal return of the last 10,000 |
|---|
One campaign that already ran with a randomised holdout. Uplift cannot be estimated from exposure alone — it needs households that were deliberately not exposed, so the difference in their behaviour can be attributed to the advertising rather than to who they were. Without that, any vendor claiming to sell you uplift targeting is selling you propensity with a better name.
This is why every audience we activate ships with an automatic holdout of a tenth, recorded in the manifest. The first campaign pays for the measurement; the second one gets to use it. A founding partner's first quarter is what makes their second quarter possible, and we would rather say that plainly than imply the model arrives fully formed.
How much holdout, concretely. The binding constraint is the size of the held-out group, not the size of the campaign — and what you can detect depends entirely on how big the effect is where you are looking. These are two-proportion power calculations at 95% confidence and 80% power, against an 11% baseline purchase rate:
| What you want to establish | Held-out households needed | Campaign reach at a 10% holdout |
|---|---|---|
| That your best quartile responds at all detecting a 2.5pp uplift | 2,700 | 27,000 |
| A usable four-way ranking the screen above | 10,800 | 110,000 |
| That a middle group is genuinely weak separating 1.0pp from zero | 16,000 | 160,000 |
| A precise overall average 0.7pp, the whole-campaign number | 32,000 | 320,000 |
Two things follow, and both are worth saying out loud because they cut against how this is usually sold. The ranking is cheaper than the average. Finding out which quarter of your audience is worth paying for needs roughly a third of the data that pinning down your overall lift to a tight interval does, because the effect you are hunting in the top group is several times larger than the effect spread across everyone. And a bigger holdout on a smaller campaign beats a small holdout on a big one: at 20% held out, every figure in the right-hand column halves. The cost of that is real — a fifth of your reach is deliberately not advertised to — which is exactly the trade a first measured quarter is buying.
Quality matters as much as volume. The holdout has to be randomised at the household level after the audience is selected, not a geography, not a store list, and not whoever the platform happened to miss — those are all correlated with purchase behaviour and will produce an uplift number that is mostly selection. Assignment seed and group sizes are recorded in the manifest for exactly this reason: so a sceptical analyst on your side can confirm the split was random rather than take our word for it.
Demonstration screen built on synthetic data with the structure and distributions of a regional grocery loyalty file. Retailer, brands and figures are invented. What is real is the method: a two-model uplift estimate trained inside the enclave against a randomised holdout, with the groups and the marginal curve computed from it.