How to Run an A/B Test When You Have Low Traffic

A/B testing low traffic websites requires a different approach from experimentation on high-volume websites. When only a small number of people visit your site or complete a conversion, it can take a long time to determine whether one version of a page performs better than another.

This is a familiar challenge for Kenyan SMEs, freelancers, online shops and service businesses. Your website might receive a few hundred visitors per month, yet you still need to decide whether to change your homepage headline, WhatsApp button, enquiry form or landing page.

The good news is that limited traffic does not prevent you from improving your website. You can run carefully planned experiments, test larger changes, observe customer behaviour and gather direct feedback. The important part is knowing which conclusions your data can support.

This guide explains how to approach A/B testing with limited traffic, when a conventional test is worthwhile, which alternatives to use and how to make decisions without mistaking random variation for a proven improvement.

What Is A/B Testing?

A/B testing is a method of comparing two versions of a page, message or experience to see how they perform against a defined goal.

• Version A: The original version, also called the control.
• Version B: The alternative version, also called the variant.

For example, a web design business might test two homepage headlines.

• Version A: “Professional Web Design Services”
• Version B: “Get a Website That Turns Visitors Into Customers”

Visitors are randomly assigned to the versions, and the business measures a predefined outcome, such as completed enquiry forms.

If Version B generates more enquiries, it may be the better option. However, the result needs enough evidence to distinguish a genuine improvement from random variation.

A/B testing is not simply publishing two designs and choosing whichever gets more clicks. It is a structured experiment with a defined hypothesis, measurement method and decision rule.

Why Is A/B Testing Difficult With Low Traffic?

The main challenge is that small samples contain more uncertainty.

Imagine your website receives 200 visits in a month. Version A receives 100 visits and generates four enquiries, while Version B receives 100 visits and generates six.

Metric	Version A	Version B
Visits	100	100
Enquiries	4	6
Conversion rate	4%	6%

Version B has a higher observed conversion rate. But the difference is based on only ten enquiries across both versions.

A few additional enquiries could change the comparison considerably. You cannot confidently conclude that Version B is better based on these figures alone.

With low traffic, you may face several challenges:

• Too few conversions to distinguish a real improvement from random variation.
• Long testing periods that delay decisions.
• Seasonal changes or campaign changes during the experiment.
• Different visitor types across the test period.
• Tracking errors that affect a large share of the small sample.
• Insufficient data to compare mobile and desktop visitors separately.

The answer is not to abandon measurement. It is to design tests that match the amount and quality of data you can realistically collect.

How Much Traffic Do You Need for an A/B Test?

There is no universal minimum number of visitors that makes an A/B test reliable.

The required sample depends on your existing conversion rate, the size of the improvement you want to detect, your chosen statistical thresholds and the design of the experiment.

A test intended to detect a small improvement generally requires more data than one intended to detect a substantial improvement.

For example, if a page converts at 2%, distinguishing a small increase to 2.2% requires much more evidence than identifying a substantial increase to 4%, all else being equal.

Conversions matter as well as visits. A site can attract thousands of visitors but still have very few purchases or qualified enquiries.

Before starting a test, estimate how many eligible visitors you can realistically collect and use a sample-size calculator or statistical testing tool to determine whether the planned experiment is practical.

If the required sample would take a year to collect, consider testing a larger change, measuring an earlier step in the customer journey or using another research method.

Step 1: Choose One Important Problem

Do not start testing simply because you want to experiment with your website design. Start with a problem that affects the business.

For example:

• Visitors reach your homepage but rarely contact you.
• People click your product pages but do not add products to their carts.
• Visitors open your enquiry form but do not submit it.
• Mobile users abandon the booking process.
• Visitors click your WhatsApp button but few become qualified leads.

Use available analytics, customer questions and sales records to identify the most important problem.

A Kenyan cleaning company might discover that visitors understand its services but cannot find the service areas or pricing information. A web design agency might find that its homepage explains its services but does not clearly show the next step.

Fixing a specific problem gives your experiment a clearer purpose.

Step 2: Write a Testable Hypothesis

A hypothesis explains what you want to change, why you expect it to help and how you will measure the result.

Use this simple structure:

Because , changing may improve .

For example:

“Because visitors may not understand what happens after requesting a quotation, changing the call-to-action text from ‘Contact Us’ to ‘Request a Free Quote’ may increase completed enquiry forms.”

This hypothesis identifies the problem, proposed change and primary outcome.

Other examples include:

Problem	Proposed change	Primary metric
Visitors overlook the contact option	Make the main CTA more prominent	Qualified enquiries per eligible session
Product pages lack useful information	Add delivery and payment details	Completed purchases per eligible session
Enquiry forms are abandoned	Remove unnecessary fields	Completed forms per form start
Service offer is unclear	Rewrite the headline around the customer benefit	Qualified enquiries per eligible session

Avoid changing the headline, images, layout, pricing and CTA simultaneously if you want to understand what caused the difference. With limited data, keep the experiment simple.

Step 3: Test Bigger Changes First

Small changes can produce small effects, which are difficult to detect when traffic is limited.

For example, changing a button from blue to a slightly different shade of blue might have little impact on conversions. Rewriting an unclear headline or simplifying a long form could make a more meaningful difference.

Prioritise changes that address a genuine obstacle in the customer journey.

Good candidates include:

• Rewriting an unclear value proposition.
• Making the primary action easier to understand.
• Removing unnecessary form fields.
• Showing delivery costs earlier in checkout.
• Improving mobile navigation.
• Making payment options and service coverage clear.
• Adding genuine customer evidence or relevant project examples.

A bigger change is not automatically better. It should still be based on an observed problem and should not introduce unnecessary risk.

Step 4: Choose the Right Metric

A test is only useful when you know what success means.

For a service website, the main metric might be completed qualified enquiries. For an online shop, it might be completed purchases.

Choose one primary metric before the test begins.

Supporting metrics can help explain the result. For example, a revised CTA may increase button clicks without increasing completed enquiries. That would suggest the change attracted attention but did not improve the full conversion process.

Useful metrics include:

Conversion rate

Conversion rate = Conversions ÷ Eligible visitors or sessions × 100

Click-through rate

Click-through rate = Clicks ÷ Impressions × 100

Form completion rate

Form completion rate = Completed forms ÷ Form starts × 100

Define the denominator consistently. A conversion rate based on sessions cannot be directly compared with one based on unique users or advertising clicks.

Also check that events are tracked correctly before launching the experiment. A test with inaccurate measurement can produce misleading results regardless of traffic volume.

Step 5: Split Traffic Fairly

In a conventional A/B test, eligible visitors should be randomly assigned to Version A or Version B, often in an approximately equal split.

Random assignment helps prevent one version from receiving disproportionately more high-intent visitors.

For example, if most Version A visitors arrive from Google search while Version B receives mostly casual social media traffic, differences in conversion may reflect traffic quality rather than the page itself.

Where possible:

• Use a testing tool that assigns visitors randomly.
• Keep both versions available during the same period.
• Avoid changing advertising budgets or targeting mid-test.
• Check that both versions load correctly on mobile and desktop.
• Ensure conversion tracking works for both versions.
• Avoid running overlapping tests that affect the same customer journey.

If you cannot reliably assign visitors to versions or record their behaviour, you may be better off conducting structured user research or a carefully documented before-and-after improvement instead.

Step 6: Decide How Long to Run the Test

Low-traffic websites may need substantially longer to collect enough evidence.

Do not choose a test duration simply because seven or fourteen days sounds reasonable. Estimate the sample you need, calculate how quickly eligible visitors arrive and consider whether the experiment will remain relevant throughout the period.

For example, a business that receives 300 eligible visits each month cannot collect 10,000 visits in two weeks. The duration needs to reflect its actual traffic.

Plan the following before launching:

• The intended sample size or statistical stopping rule.
• The primary metric.
• The minimum improvement that would matter commercially.
• The test duration or review conditions.
• Any situations that would require stopping for technical or business reasons.

Avoid checking the results every day and stopping as soon as one version appears ahead. Repeatedly checking conventional statistical results and stopping opportunistically can increase the risk of a false positive.

If your tool supports a properly designed sequential test, follow its stated decision rules. Do not assume that looking at results more often automatically makes a test faster or more reliable.

Step 7: Interpret the Results Honestly

At the end of the experiment, the outcome will usually fall into one of three categories.

Outcome A: There is convincing evidence that one version performs better

If the experiment was designed correctly and the evidence supports a meaningful improvement, consider adopting the stronger version.

Still check secondary metrics. A headline that increases enquiries but attracts unsuitable leads may not improve business performance.

Outcome B: The results are inconclusive

An inconclusive result does not mean the versions perform identically. It means the available evidence does not establish a clear difference under the chosen method.

With low traffic, this is common.

You can collect more data if the test remains worthwhile, test a larger change or use customer research to identify the next improvement.

Outcome C: The new version performs worse

If the evidence indicates that the variant harms the primary metric or an important business outcome, retain the original version or investigate why the change failed.

Check for technical problems, poor mobile presentation and differences in traffic quality before drawing a final conclusion.

The key is to avoid turning a small numerical lead into a claim that one version has definitively won.

What to Do When You Do Not Have Enough Traffic

Sometimes a conventional A/B test is not the best use of a small business’s time and resources. You can still improve the website through other methods.

1. Run usability tests with real people

Ask a small number of people who resemble your target customers to complete a specific task.

For example, ask them to find a service, understand the pricing information and submit an enquiry.

Observe where they hesitate, misunderstand the page or struggle to complete the task.

A handful of usability sessions cannot establish a statistically reliable conversion-rate improvement, but they can reveal obvious usability problems that deserve attention.

2. Use customer interviews

Speak to existing customers and qualified prospects about what influenced their decisions.

Ask questions such as:

• What made you consider this business?
• What information did you need before contacting us?
• Was anything confusing on the website?
• What almost stopped you from enquiring?
• What other options did you consider?

Use the answers to develop hypotheses, not as proof that every visitor shares the same opinion.

3. Review analytics and behaviour patterns

Look at the steps visitors take before converting or leaving.

For example, if visitors reach a quotation form but rarely complete it, inspect the form length, error messages, mobile usability and clarity of the next step.

Tools such as Google Analytics 4 can help measure configured events and traffic behaviour. Heatmaps and session recordings may also help identify usability issues, subject to appropriate privacy controls and consent requirements where applicable.

4. Improve one thing at a time and track before and after

If a full randomised experiment is not feasible, make a clearly documented improvement and compare suitable periods before and after the change.

Record the dates, traffic sources, conversion counts, marketing spend and any other significant changes.

This method is weaker than a properly randomised experiment because seasonality, campaigns and other external factors can influence the result. Treat the findings as directional evidence rather than proof that the change caused the outcome.

5. Measure an earlier step in the customer journey

Purchases may be too rare to measure meaningfully, but earlier actions may happen more often.

For an online shop, you could study product-page engagement or add-to-cart activity. For a service business, you might measure form starts or clicks to reveal contact details.

Earlier actions can help diagnose friction, but they are not substitutes for purchases or qualified leads. Make sure the action is genuinely related to the business goal.

6. Combine genuinely similar pages when appropriate

If several service pages serve similar audiences and have the same conversion goal, you may be able to design an experiment across those pages.

However, do not combine unrelated products, traffic sources or customer types simply to increase the sample. Pooling is only useful when the pages and audiences are sufficiently comparable and the experiment is designed to account for the differences.

A Practical Example for a Kenyan Service Business

Imagine a Nairobi-based web design agency receives 400 eligible website sessions per month. Its current enquiry conversion rate is 2.5%.

That produces approximately ten enquiries per month.

The agency wants to test a new homepage headline, but splitting the traffic between two versions would produce only around five enquiries per version per month if performance remained similar.

That is a very small number of conversions. A short experiment would probably not provide strong evidence of a modest improvement.

Instead, the agency could take the following approach:

Week 1: Diagnose. Review analytics, inspect the mobile homepage and ask a few target customers to explain what the business offers.

Week 2: Form a hypothesis. The team finds that the headline describes web design services but does not explain the business outcome. It proposes a more specific customer-focused headline.

Week 3: Prepare the change. The team checks that the new headline accurately describes its service, improves the CTA if needed and verifies that enquiry tracking works.

Week 4 onward: Measure. The agency records enquiries, qualified leads, traffic sources and changes in marketing activity. It interprets early results cautiously and continues monitoring.

If enough traffic becomes available, the agency can run a properly planned randomised A/B test. If it remains too low, usability research and documented sequential improvements may be more practical.

This approach does not guarantee an increase in conversions. It gives the agency a structured way to improve its website while being honest about what its data can establish.

Which Tools Can Help With Low-Traffic Testing?

The right tool depends on whether you need to run a randomised experiment, understand user behaviour or measure outcomes.

Tool or method	Best use	Important consideration
Google Analytics 4	Track website events and traffic patterns	It does not itself provide a complete native website A/B testing workflow
Google Tag Manager	Configure and manage many tracking tags	Incorrect event setup can distort results
A dedicated A/B testing platform	Randomly assign visitors and evaluate variants	Check pricing, traffic limits and integration requirements
Heatmaps and session recordings	Identify usability problems	Use appropriate privacy and consent safeguards
Google Sheets or Excel	Record experiments and compare results	Manual comparisons do not establish statistical significance
Customer interviews and usability tests	Understand confusion and barriers	Qualitative feedback does not measure conversion lift directly

Google Optimize was discontinued on 30 September 2023, so avoid tutorials that present it as a currently available testing tool. Google documents its sunset and points to third-party experimentation integrations. See the Google Analytics notice about Google Optimize and Google Search Central’s guidance on website testing.

You do not need to buy an expensive tool before you have identified a meaningful problem and confirmed that the necessary tracking is in place.

Common Mistakes to Avoid

Testing too many changes at once

If you change the headline, page layout, images and CTA together, it becomes difficult to understand which change influenced the outcome.

Declaring a winner too early

A temporary lead based on a few conversions is not enough to establish a reliable winner.

Ignoring the quality of conversions

A version that produces more enquiries may generate fewer paying customers if the leads are less relevant.

Changing the campaign during the test

Adjusting targeting, budget, offer or pricing during an experiment can make the comparison harder to interpret.

Treating an inconclusive test as a tie

The data may simply be too limited to detect a meaningful difference. Record the uncertainty and decide whether additional testing is worthwhile.

Using outdated tools

Before following an A/B testing tutorial, verify that the tool still exists and supports the workflow being described.

Forgetting the customer experience

Do not make changes purely to increase clicks if they create confusion, hide important information or encourage low-quality enquiries.

A Simple A/B Testing Plan You Can Reuse

Use this checklist before launching an experiment.

• Identify a specific conversion problem.
• Write a clear hypothesis.
• Choose one primary metric.
• Define the eligible audience and denominator.
• Verify conversion tracking.
• Estimate the sample size and likely duration.
• Decide how results will be evaluated before the test begins.
• Keep other campaign conditions as consistent as practical.
• Record the results, including conversion counts.
• Document the decision and the next action.

If the test is unlikely to collect enough data, switch to usability research or a carefully documented improvement plan instead of pretending that a small sample can answer a question it cannot.

Frequently Asked Questions

Can I run an A/B test with 100 visitors?

You can technically compare two versions with 100 visitors, but the result will usually be highly uncertain unless the effect is very large and the circumstances are favourable. Whether the sample is adequate depends on the baseline conversion rate and the improvement you want to detect.

How long should an A/B test run on a small website?

There is no fixed duration. Estimate the required sample size and compare it with your actual eligible traffic. A test that cannot gather enough evidence within a commercially useful period may not be worth running in its current form.

Is before-and-after testing the same as A/B testing?

No. A conventional A/B test randomly compares versions during the same period. A before-and-after comparison evaluates results from different periods, so seasonal changes, advertising activity and other factors may influence the outcome.

Should I use Bayesian testing for low traffic?

Bayesian methods can help you express uncertainty and evaluate the probability of different outcomes using a defined model and prior assumptions. They do not create information that the data does not contain. A small sample can still produce a wide range of plausible outcomes, so interpret results carefully.

What should I test first on a low-traffic website?

Start with a problem that has clear evidence behind it. A confusing headline, a difficult enquiry form, unclear pricing or a broken mobile journey may be more important than minor cosmetic changes. Use customer research and analytics to prioritise the most meaningful issue.

Do I need a paid A/B testing tool?

Not necessarily. A paid platform may make random assignment and analysis easier, but it cannot compensate for inadequate data or a poorly designed experiment. Start with the measurement and research methods you can maintain reliably, then evaluate specialist tools if your needs justify them.

Conclusions

Low traffic changes how you should experiment, but it does not mean your website must remain unchanged. The most effective approach is to match the method to the question and the amount of evidence available.

Run conventional A/B tests when you can collect enough data. Test meaningful changes rather than tiny cosmetic details. When a reliable experiment is impractical, use customer interviews, usability testing and carefully documented improvements to discover what needs attention.

The goal is not to declare a winner after every experiment. It is to make better decisions, learn from real customer behaviour and gradually improve the results your website produces.

Need a website that turns more visitors into meaningful enquiries? Elon Designs can help your business improve its website structure, messaging and user experience. Visit Elon Designs to explore how a stronger digital foundation can support your marketing goals.