How to Run a Price A/B Test the Right Way

A price test you can actually trust needs three things working together: a price change large enough to produce a real signal, a duration long enough to smooth out normal day-to-day noise, and enough traffic to reach genuine statistical significance rather than a result that looks meaningful but isn't. Most small stores get one or two of these right and skip the third without realizing it. This guide covers how long to run a price test, how much to actually change the price, what statistical significance means in plain terms, the real difference between a true A/B test and a simple before/after price change, and whether you need a developer to do any of this. For the trust, fairness, and legal considerations around showing different prices to different customers, price survey vs price testing covers that ground in depth; this post focuses on getting the test itself right.

How Long Should You Run a Price Test

Two weeks is the most commonly cited minimum, and it's a reasonable floor for most stores with meaningful traffic. Guidance varies past that point: some sources recommend running through at least one full business cycle, generally 2 to 4 weeks, to smooth out the normal difference between weekday and weekend shopping behavior, while lower-traffic stores are often advised to run 4 to 8 weeks simply because they need more calendar time to accumulate enough orders to say anything with confidence. There's no single universal number here, the real constraint isn't the calendar, it's whether you've accumulated enough orders to reach significance, which is a volume question as much as a duration one.

Is Your Result Significant Enough to Trust

Statistical significance is really just a measure of how likely your result is to be a real effect rather than random noise. A 95% confidence level, standard for most ecommerce tests, means there's roughly a 5% chance the difference you're seeing is a false positive rather than a genuine response to the price change. For a high-stakes decision like a permanent price change, some practitioners recommend tightening that to 99% confidence before acting, since the cost of being wrong is higher than it is for a smaller UI test.

Here's where a lot of guidance aimed at large ecommerce brands stops being useful for a small store: some sources cite sample-size thresholds like 30,000 visitors per variant with at least 3,000 conversions before a result counts as valid. That number describes a high-traffic enterprise store, not a typical independent Shopify or WooCommerce seller, and treating it as a universal requirement would mean most small stores could never run a valid price test at all. The more practical bar for a smaller store: at minimum, aim for 100 or more conversions per price variant before drawing a conclusion, and treat anything below that as directional rather than a settled result, similar to how a low-response survey should be read as a rough signal, not a precise number.

How Much to Change the Price

Test a meaningful move, not a token one. Guidance converges around a 5% to 20% price change as the range large enough to produce a detectable shift in customer behavior; smaller moves risk getting lost in normal day-to-day noise regardless of how long you run the test. A $50 product tested at $52 is unlikely to tell you much of anything useful. The same product tested at $55 to $60 gives you a real chance at reading an actual response.

Volume matters as much as the size of the move. A meaningful price change on your highest-volume product will reach a trustworthy sample size faster than the same percentage move on a slow-selling SKU, simply because more orders accumulate in the same calendar window. If you can only run one test at a time, run it on a product with real, steady sales volume rather than a thin one, even if the thin one is the product you're most curious about.