Why incrementality mistakes are expensive
Incrementality asks whether QR-driven conversions would have happened anyway. It is the hardest and most honest question in QR measurement, because a scan that merely replaces a typed URL adds no new value.
Mistakes in measuring or managing incrementality are expensive because they corrupt decisions. A wrong number does not just sit in a report; it moves budget, kills working placements, and rewards failing ones. Incrementality tests are expensive to run well and cheap to get wrong. A sloppy holdout is worse than no holdout. The good news is that most mistakes are repeatable and preventable. They come from a short list of causes, and a short checklist catches nearly all of them.
Mistake 1: Measuring with an unstable definition
The single most damaging mistake is changing the definition mid-campaign. Counting unique scanners one month and all scans the next makes the trend meaningless, even though both numbers are called incrementality. Fix it by writing the definition down: Incremental conversions = conversions in the exposed group minus conversions in a comparable holdout group. Incremental lift = incremental conversions divided by holdout conversions, multiplied by 100. Keep that definition stable, and if you must change it, note the change in the report and restart the trend from that point rather than pretending the old and new numbers are comparable.
Mistake 2: Comparing unlike placements
A code at a checkout, a code on a poster, and a code in an email serve different moments and different intents. Comparing their incrementality as if they were equal leads to confident, wrong conclusions, such as killing a placement that was actually performing well for its context. Fix it by segmenting before judging: compare each placement against its own past, and only compare across placements when audience and context are genuinely similar.
Mistake 3: Ignoring the denominator
People remember the numerator and forget the denominator. Scans rose, so the campaign improved, or so the story goes, until someone notices exposures rose faster and incrementality actually fell. Fix it by always reporting both sides of the ratio and the raw counts behind them. A percentage without its counts is a claim without evidence.
Mistake 4: Reacting to noise
Daily numbers swing for reasons that have nothing to do with quality: weather, weekdays, holidays, and one large group passing a poster. Reacting to a single day means changing working placements on the basis of noise. Fix it by deciding the review cadence in advance, using comparable periods, and requiring two consecutive out-of-range readings before acting.
Mistake 5: Blaming the code for a downstream problem
Treating every scan as incremental by default. and Running a holdout in a different season and calling the gap causal. both appear in reports that blame the QR code for problems elsewhere in the journey. Before redesigning the code, check the steps around it: placement, lighting, landing page, form, and offer. The code is often the most visible part of the journey and the least guilty.
Putting this into practice
Audit your current incrementality reporting against the five mistakes above. For each one, write down whether it applies to your reports and the one fix you will make this week. Then re-issue the last report with the fixes applied: stable definition, segmented comparisons, both sides of the ratio, a sensible review cadence, and a check of the surrounding steps. The corrected report may tell a different story, and that different story is the point.
Common mistakes to avoid
- Treating every scan as incremental by default.
- Running a holdout in a different season and calling the gap causal.
- Changing the definition mid-campaign without noting it.
- Judging a placement against an unlike placement.
- Reacting to one day of data instead of a full comparable period.
Quick checklist
- Write the definition down and keep it stable.
- Segment by location, audience, and test period before judging.
- Report both sides of the ratio and the raw counts.
- Use comparable periods and a fixed review cadence.
- Check the steps around the code before blaming the code.
- Note any method change alongside the numbers it affects.
Related tools and guides
- Design a fair comparison with the [QR A/B test planner](/qr-a-b-test-planner), including sample size and decision rules.
- Check a whole batch of destinations at once with the [QR batch validator](/qr-batch-validator) before printing.
- Tag every destination consistently with the [UTM builder](/utm-builder) so each scan can be traced to its placement.
- Run unfamiliar destinations through the [QR safety checker](/qr-safety-checker) before sharing a code widely.
- Match printed size to viewing distance with the [QR size calculator](/qr-size-calculator) before you approve artwork.
- Test every code in the real location with the [QR scanner](/qr-scanner) on more than one phone.
- The [blog](/blog) collects more guides on sizing, printing, security, and measurement for QR codes.
