Incrementality Testing: Measuring the True Impact of Your Marketing

Why Platform-Reported ROAS Is Often Wrong

Every ad platform — Google, Meta, LinkedIn — is heavily incentivised to claim credit for conversions. When a customer purchases after seeing a Google Ad that ran alongside an email they also received and a brand search they conducted, Google reports the purchase as a conversion. Meta may also report it as a conversion. And your email platform claims it too. The reality is one purchase; the reported conversions may be three.

Incrementality testing is the methodology that cuts through this by measuring how many conversions would have happened anyway without the marketing activity in question.

How Holdout Tests Work

The most rigorous incrementality test is a geographic or user-level holdout:

1. Randomly assign users or geographic areas into two groups: test (receives the ad) and control (holdout — does not receive the ad)
2. Run the campaign normally to the test group while intentionally withholding it from the control group
3. Compare conversion rates between groups after the campaign period
4. The difference in conversion rate is the incremental lift attributable to the campaign

Incremental ROAS = (Revenue from test group - Revenue from control group × test/control ratio) / Campaign spend. This is the true return on that specific ad spend.

Platform-Level Holdouts vs. Channel-Level

• Platform holdout: Turn off Meta Ads entirely for a random 10% of your customer audience. Measure whether their purchase rate declines. If it declines significantly, Meta is driving incremental sales. If not, you could reallocate that budget.
• Campaign holdout: Within Meta Ads, create a holdout audience (available in Meta's Conversion Lift tool). More granular — measures the specific campaign's incrementality.

Geo Holdout Tests for Full-Mix Incrementality

Geographic holdouts pair matched market areas and compare performance when marketing runs in some markets but not others. This is particularly useful for measuring TV, radio, or out-of-home impact alongside digital channels. Our marketing analytics service designs and analyses incrementality tests for marketing teams that need to know their true ROI. Contact us to plan your first incrementality test.