A/B testing with low traffic
Most A/B testing advice is written for sites with millions of visitors. If your page gets a few thousand visits a month, you can still test, but you have to test differently. This guide says what works on a small site, what to skip, and when a test is not worth running at all.
Can you A/B test with low traffic?
Yes, with limits. A small site can detect big differences, not small ones. If a change might lift conversions by 30 % or more and 5 % or more of visitors already complete the goal, a page with a few thousand visits a month can usually show it within a few months. With a rarer goal, such as a purchase, the same test takes most of a year when 2 % complete it, and more than a year at 1 %. If the change might lift them by 2 %, no test on a small site will see it, and running one only wastes time. Use the estimate in How long to run an A/B test before you start.
Test big changes
Small tweaks (a button colour, one word in a heading) produce small differences. On a small site, test changes that could matter a lot:
- a different headline and promise, not a reworded one
- a shorter or longer page
- a different offer, such as a demo instead of a trial
- a form with fewer fields
- a redirect test that compares two different page designs
Pick a goal more visitors complete
A goal that 20 % of visitors complete gives an answer far sooner than one that 1 % complete. If purchases are rare, test the step before: a click on "Add to cart", "Book a call" or "Get a quote". Keep an eye on the later step too: in Twinpage you can track it as an extra metric next to the main goal, without slowing the test down.
Test fewer versions
Every variant takes its own share of the visitors. On a small site, test the original against one variant. Add more only when you have the traffic.
Test your busiest page
Run the test where your visitors already are, usually the home page or the landing page your ads point to. A test on a page with 200 visits a month will rarely finish.
Run it long enough, and don't peek with the wrong maths
Low traffic tempts you to stop early. Run whole weeks, and either wait for the planned number of visits or use a tool with a sequential test, which is built for checking early. Twinpage uses one and waits for at least 1,000 visits and 25 conversions per version before it calls a winner.
When not to A/B test
If the estimate says more than a year, a test will not give you an answer. Talk to five customers, watch how people use the page, or make the change you believe in and compare the weeks before and after, knowing that comparison is rougher than a test. Come back to testing when the traffic grows.
Frequently asked questions
How much traffic do I need to A/B test? It depends on your conversion rate and how big the change is. As a rough guide, with a 5 % conversion rate each version needs about 4,500 to 6,800 visits to show a 30 % lift. The table in How long to run an A/B test has more examples.
Is a result from a small site reliable? It is as reliable as the test behind it. A proper significance test with enough conversions per version gives a result you can act on, even on a small site. A result from a few dozen visitors is not.
Should I test on mobile and desktop separately? Not on a small site. Splitting the traffic further makes each test slower. Test everyone together first.
Testing with Twinpage
Twinpage is A/B testing for marketing sites: one script tag, a visual editor, one main goal per test and a verdict you can trust, with prices you can read before you start.