Why improving incrementality is worth the effort
Incrementality asks whether QR-driven conversions would have happened anyway. It is the hardest and most honest question in QR measurement, because a scan that merely replaces a typed URL adds no new value.Incrementality tests are expensive to run well and cheap to get wrong. A sloppy holdout is worse than no holdout.
Diagnose before you change anything
Improvement starts with diagnosis, not with redesign. Before changing a code, split incrementality by location, audience, and test period and look for the weakest segment. The weakest segment is where a fix buys the most, because it holds the largest pool of lost value. Also check the steps around incrementality: a problem upstream, such as poor placement, can masquerade as a problem with the code itself, and a problem downstream, such as a slow landing page, can waste scans that were earned fairly. Fix the constraint, not the symptom.Write down your diagnosis in one paragraph before you touch anything. State what you observed, which segment is weakest, and what you believe is causing it. This paragraph is your hypothesis, and it keeps the work honest: if the fix does not move the number, the hypothesis was wrong, and you learn something instead of merely changing things.
Levers that move incrementality
- Test with a real holdout instead of assuming every scan is incremental. - Keep the test running long enough to cover normal weekly variation. - Spend more where incrementality is proven, not where scans are highest.Work through the levers in order of expected impact per unit of effort. A lever that costs nothing and affects every scanner beats a lever that costs a print run and affects one placement. For each lever, state the change, the cost, and the segment it should move, then implement only that change. Bundling several changes together feels productive, but it destroys your ability to learn which one worked.
Test the change properly
Incremental conversions = conversions in the exposed group minus conversions in a comparable holdout group. Incremental lift = incremental conversions divided by holdout conversions, multiplied by 100.Illustrative example numbers, not real-world data: an exposed group produces 45 conversions and a comparable holdout group produces 30, giving 45 - 30 = 15 incremental conversions and 15 / 30 x 100 = 50% incremental lift in this example.
Run the change as a comparison wherever you can: the old version against the new version, on comparable placements, over the same kind of period. Keep everything else stable, and decide in advance how long the test runs and what counts as a win. A test that ends the moment one variant pulls ahead is not a test, it is a reaction. If traffic is too small for a formal test, change one thing, record the before and after numbers, and treat the result as suggestive rather than proven.