Multivariate testing: the complex process for optimizing the conversion rate

An optimized conversion rate is the goal of every developer or web project operator. First-class content and products, as well as constantly increasing traffic, are usually promising signs for upcoming stores and companies. Good as these might sound, they are useless if they don’t result in transactions, clicks, or acquisitions from completed forms. While the number of visitors can be influenced by quite different measures (social media marketing, SEO, SEA, link building, etc.), the amount of conversions is mainly due to two factors: how much trust the user has in the store or website, and the web project’s user-friendliness, which is also referred to as usability. In order to consistently improve the latter, multivariate testing is becoming increasingly popular among developers and marketers. The advanced testing method, which is even more complex than the often-performed A/B testing, can detect weaknesses among the elements being checked. These can then be rectified to make it easier for the user to access the content. If visitors enjoy their experience they will spend more time in the store or on the website and are more likely to carry out the desired action (conversion).

The following example illustrates the approach to multivariate testing: the web page of a certain product should be optimized since the description and image used were selected as decisive test criteria. The analysis procedure is intended to determine whether and to what extent the two different product descriptions A and B (as well as the various product images 1 and 2) influence the conversion rate. For this purpose, the images and descriptions are combined to form the following combinations:

• Description A, Image 1
• Description A, Image 2
• Description B, Image 1
• Description B, Image 2

This simple example shows that there is a maximum of four different combinations – the number of possible combinations rises with each additional variable. The four different versions are now presented to potential customers who access the store’s website. The entire traffic is distributed evenly among the four variants. The conversions (completed transactions) are analyzed during the review period using an analysis tool such as Webtrends Optimize. This is so that a conversion rate can be calculated for all four variants after the multivariate test is completed. The bigger the amount of traffic and the longer the observation period, the more significant the results.

Compared to usability tests (which are undertaken in the project’s development phase), multivariate tests have the advantage that a much larger number of participants can be included seeing as these tests are run once the website, app, or store is online. These are known as quantitative research methods. Although multivariate testing seems extremely complex, it is actually quite easy when the right tools are used. The various test pages are quickly configured and implemented via JavaScript code snippet in the web project. The results are shown in real-time in clearly arranged tables so that it’s easy to see which combinations promise the greatest success.

Unlike A/B testing, multivariate testing isn’t limited to two versions; there’s actually no limitation. This facilitates the verification of several different elements and also provides the marketer with comprehensive insights into how various components interact with each other. Multivariate tests don’t just reveal which combinations influence the conversion rate in a positive or negative way when it comes to individual cases; they also provide a concrete, statistically proven picture of which components contribute to the overall success of a web project and in which ways. The knowledge gained during multivariate testing can also play a major role in the development of future projects.

High traffic is mandatory for a multivariate test to obtain results that are as accurate as possible since the traffic is split and distributed to at least four test examples. An additional problem with this testing method is when one or more of the tested variables have no influence on the conversion goal – especially when it comes to interpreting the results. Here, multivariate testing can quickly end up being the wrong choice because it ends up complicating the evaluation process unnecessarily. In this case, a simple A/B test might have sufficed.

The best way to test the usability of your web project depends on a number of different factors, with the amount of traffic playing the biggest part. If your web project is only recently up and running and you haven’t had many visitors as of yet, multivariate testing won’t provide you with reliable results on the success of each variant. In such a case, it is recommended to test the functionality of individual variables in consecutive A/B tests. If on the other hand, your web presence has high traffic, multivariate tests are the better option since you save time and effort.

A pre-requisite for both methods is that you have to have formulated clear hypotheses and result quantities for the tested elements. Otherwise, the test results will be difficult to interpret. In contrast to the usability analysis during the development process, you must also expect the conversion rate to deteriorate temporarily during this live test. So when it comes to testing out new ideas or variants, multivariate tests are neither efficient nor adequate, and A/B testing is often too limited. Preliminary experiments on a smaller scale and with clearly formulated questions are a much more effective and less risky solution.

If your web project has enough traffic and you decide to perform multivariate testing, you shouldn’t see this as a carte blanche to create as many variables or versions as possible. For an optimal result, it is advisable to proceed as strategically as possible and only have the main opponents compete against each other by pre-selecting which ones these are. In order to play it safe, you can even check the test result with a subsequent A/B test. However, you should be aware that using one of these testing methods doesn’t guarantee an increase in the conversion rate, but merely gives you ideas for what could increase it.