How to Run a QR A/B Test That Actually Means Something
An A/B test compares two versions of the same QR placement and asks one question: which one performs better, and by how much. The comparison only means something if the two variants are tested fairly, on comparable traffic, with a sample large enough to distinguish a real difference from random noise. Most QR tests fail on one of those three conditions. They run for a weekend, they compare a poster in a busy entrance with a poster in a quiet corridor, or they declare a winner because one variant got three more scans. This planner exists to prevent those mistakes by fixing the sample size, the metric, and the decision rules before the test starts.
Start with a single change. Variant A is the current version; variant B changes one thing, such as the headline, the code size, the call to action, or the placement. Changing three things at once makes the result uninterpretable: if B wins, you will not know which change caused it, and you will not know what to keep. The success metric must also be single and measurable. Scan rate is the natural metric for a QR test, but conversion rate or revenue per exposure may matter more if the goal is sales rather than scans. Pick the metric that matches the business goal, write it down, and do not switch metrics halfway through because another one looks better.
Sample size is where the arithmetic matters. The smaller the improvement you want to detect, the larger the sample you need, and the relationship is punishing: halving the minimum detectable effect roughly quadruples the required sample. That is why the planner asks for the smallest improvement worth acting on, not the improvement you hope for. If the honest answer is that only a large improvement would justify switching, the sample stays manageable. If you insist on detecting a tiny effect, the planner will tell you the test needs more traffic than the placement can deliver, and the correct response is to run the test longer, pick a higher-traffic placement, or accept a larger minimum effect.
Fairness in traffic is the condition people overlook. The two variants must see comparable audiences, in comparable places, over the same days. Running variant A on weekdays and variant B on weekends confounds the test with the calendar. Placing A at eye level and B behind a door confounds it with visibility. The cleanest design for physical QR tests is alternating placements in the same location on the same days, or two matched locations with similar foot traffic. Keep the split as even as you can, and record the exposures per variant, not just the scans, because a scan count without its denominator is meaningless.
The decision rules must be written before launch, because after launch everyone is tempted to move the goalposts. The rules on this planner are deliberately strict: adopt B only if it beats A by at least the minimum detectable effect at the chosen confidence; keep A if the difference is smaller; stop if B is worse. Do not peek daily and stop the moment a result looks good, because early results bounce wildly and stopping at a peak guarantees an exaggerated winner. Run the test to its planned sample or its planned duration, then decide once, and write the result down with the numbers so the next test starts from evidence instead of opinion.