Essay · 2 February 2026
Geo-holdouts for grocery apps that actually ship
A holdout that exists only in a slide is not a test. Grocery delivery makes that painfully obvious.
United States DMA maps do not transplant onto the United Kingdom. Commuter overlap between Leeds and Bradford, or between Manchester and a ring of towns, will leak treatment into control if you pick cities by vibe. We pair on pre-period order density, weather sensitivity, and whether national TV already saturates both cells.
Legal review matters because pausing paid social in a city can look like a trading decision. Write the intent as a measurement experiment with a start date, a stop date, and a restoration of spend. Counsel in our Yorkshire example asked for a sentence that the holdout was not a punishment of a local franchise. We added it. The test shipped.
Agencies will try to “protect” the control with brand search still running. That is a different experiment. Either search is in both cells or you are measuring a cocktail. We prefer a dirty test you can describe over a clean test nobody ran.
When results arrive, resist the national multiplier. The incrementality ratio belongs to those city pairs, those weeks, that creative. If TV leaked, say so in the first paragraph of the memo. The Signal Mapping Lab grades that honesty harder than a pretty confidence interval.