How to Define your A/B tests

Combining methodology with common sense

If you're interested in launching an A/B testing project, there are two possible scenarios to explore:

• you have no idea about what should be tested: In this case, contact a consulting firm that will carry out the analytical and user experience audits that are required to define your test roadmap before you begin. However, even if you do so, you still need to put yourself in your visitors' shoes, and there are several tests that you can carry out based on the examples provided next.

• You already have an opinion or an intuition, or even an existing preference for a particular approach. Be daring and follow your intuition! Even though this approach doesn't exempt you from following a more rigorous methodology, starting with the most obvious hypotheses is something we can only encourage wholeheartedly. Be careful though, as haphazard testing without defined objectives won't work. If you take this approach, your A/B tests won't be clearly differentiated. However, if you already have some fairly specific ideas, it is likely that you will identify gains (or losses) in your conversion rates.

Three playgrounds

There are three playgrounds that you can use to create and test new variations. You can specialize in just one aspect, but market-leading conversion rates can be achieved by mastering all three.

Your Product

Your range of products and/or services is your area of specialist knowledge, where your expertise is likely to be stronger than that of any outside organization. A/B testing gives you a low-cost tool to test the various hypotheses concerning the presentation of your offer:

• The value proposition: Is it clear, coherent, and consistent between pages? Should you highlight the quality of your product, its benefits, or its price? Does your website convey the image you want to present?

• The information provided: Do you have the right level of details? Should you share more information, or should you pare it back to the essential? Have you eliminated the background noise that could distract visitors' attention?

• The format: Could a short video be more effective than a long-drawn-out presentation?

• The product or service itself: Does one product or service cannibalize the sales of another? Is the market pricing appropriate?

• Customer guidance: Is the visitor guided and reassured by the right messages? Are promotional offers clearly highlighted?

Your data

Perhaps without even being, aware, you already have a considerable volume of information about the behavior of your visitors, aggregated in your web analytics tool. 

If an abnormally high bounce rate is observed from a landing page, this landing page is an obvious candidate for a series of A/B tests to optimize its performance. The same applies to a sales funnel with a high abandonment rate. 

In the absence of high-level analytical capabilities within your organization, make use of the services of a specialist agency to capture, analyze, and visualize the data with acceptable turnaround time. Do this before starting to design your A/B tests. In any case, it is essential to support your A/B tests with every actionable data available.

Your User Interface

The behavior of your visitors changes very rapidly, as do trends within the field of UX. A user experience audit is often the right solution to identify the underperforming area of your website, as a decision made during a particular year might no longer be the right decision a year later. A careful analysis of the path followed by customers (behavior flow, browsing, etc.) also requires further, detailed work.

Without being a UX expert, it is nonetheless possible to carry out an enormous range of tests on all elements of your website that could influence your visitors and have a positive impact on your conversion rates. These include:

• The design, position, and general look and feel of text, text blocks, and CTAs: their size, font, color, alignment, etc.

• The wording, particularly that of CTAs: Users will understand and react differently to a CTA that says "Buy now" instead of "Buy this product" or "Add to cart", and this can lead to surprising differences in conversion rates.

• Forms: User sign-up forms are one of the areas that benefit the most from A/B testing. The user experience and the appearance of these forms are vital to maximizing your users' desire to sign up for your service. The layout, text, field length, and placement of labels should all be tested in order to observe which format maximizes sign-up rates.