So, you want to get the most of your website doing conversion rate optimization via A/B testing. The ideal notion about it will be something like provide an A/B test and get 25% lift in conversions after changing CTA on the landing. But in the reality, everything is not so literally. There are a lot of myths that interfere the proper way of performing A/B testing.
Today I’ll share the top-10 of the most common myths and, for each of them, what the truth really is.
Myth #1 – Conversion Rate Optimization Occurs Due to Guessing
The myth. Since you work in your niche, you know your customers perfectly, like no one does. And that’s the reason why you don’t need any special tool or method to optimize your online presence. You just know the best what to offer to your visitors, where, when, and how to do it.
The truth. Actually, A/B testing doesn’t mean you need to give up your knowledge about your business or customers. It only means you’ll apply your predictions via methodology of making informed decisions on how to improve your online business. Anyway, the process of improving your conversion rate will start with defining “the weak zones” of your website. And after defining of weaknesses you’ll have to make the grounded choice how to deal with it. A/B and MVT testing is just the scientific approach of making right decisions about your website’s improving.
Myth #2 – A/B Testing Harms Your SEO
The myth. As we all know, in the process of A/B testing the special A/B testing tools show their visitors almost the same variations of the content to define the best performing one. Also, we know that Google (and other search engines) always penalizes duplicated and plagiarized content. So, obviously, SEO will be spoiled if you ab-test your website.
The truth. As we’ve already written in our expanded article about the relationship between SEO and CRO, there is no danger for your website if you’re improving your website using the methodology of A/B or MVT testing. Google wrote the full list of guidelines for providing proper A/B experiments and if you’re using the proven A/B testing tools there is no danger for you. Maxymizely follows all of Google’s guidelines, so you’re absolutely safe with us.
Myth #3 – A/B Testing is Not Easy to Use Due to the High Cost of Implementation
The myth. A/B testing is a very expensive method of conversion improving as it uses a lot of data from visitors that is hard to analyze, and also there is a need to hire a designer and a coder to start working with this.
The truth. A/B testing isn’t so difficult or expensive. There are a lot of automated all-in-one tools which allow designing your variations intuitively, launching your A/B tests and analyzing the results without any third-side help. You should only listen to the given recommendations/tips and apply them. One of such tools is Maxymizely.
Myth #4 – You Should Only Measure 1 Important Conversion Rate Metric at a Time
The myth. While running the A/B test you should focus only on one metric as a goal not to get lost in a pile of different figures. If you’re testing a button you should only consider the winner variation that shows the best CTR for the tested button.
The truth. The proper way of holding the A/B experiment is to consider the gained A/B testing goal in its relation to the main business goal. Let’s imagine that the winner variation with the red button performed better at the first stage (after clicking this button), but at the final stage, this variation leaded to the lower conversion into leads. So, you can see that the winner variation didn’t bring your business real profit and you need to change your focus slightly in order to achieve your business goal.
Myth #5 – You can stop the test when the results look good
The myth. Right after the start of your A/B testing experiment you may observe the process of increasing the page views for every of your tested variations. It will allow you to see almost in real time which of your variations perform better and to choose the winner after you’ve seen an essential difference in conversion rate that is acceptable for you.
The truth. Don’t be so fast! Yes, it’s true that the most of A/B testing tools will start to show you the changing statistics right after the launching of the experiment (of course, if you have enough traffic to see the changing numbers of visits, reached goals, etc. in real time). But, you shouldn’t forget about statistical significance which is necessary to consider in A/B testing experiments to get reliable results. There is a good publication of Peep Laja about statistical significance and validity in A/B testing you should learn as a must. There is a reason you should run the A/B testing for a certain amount of time and through a specific number of visitors. It provides you with relevant data, which will guide you to better decisions. If you just stop the process halfway, you might get an inaccurate statistics in your hands.
Myth #6 – Everything on your website should be tested
The myth. To get really good, reliable results you should A/B test everуthing on your website. Any change you can test on your website should be tested. It even doesn’t matter how big are your changes and what is the expected growth caused with your conversion improvement. If somebody has done it already and proved his work with numerous case studies why shouldn’t you do the same? Testing every component on a website will bring to incredible results in business!
The truth. Of course, it’s easy to find a lot of case studies about A/B testing of every single component existing at a website but there should be another sequence of A/B tests. Firstly, you have to examine your website by analyzing your conversion funnel. The detailed article on how to do it is under the link: How to discover website areas you should improve via A/B testing. Great case studies can be wonderful sources for your inspiration, a source of new ideas to test. But A/B testing isn’t about testing everything all the time. You should build and implement your own A/B testing strategy, which will be based upon your business goals and your conversion funnel with its own “pain spots”.
Myth #7 – Studying and Copying A/B Testing Best Practices is All That You Need to Succeed in Conversion Rate Optimization
The myth. If you’re studying the best practices from some big and reliable companies you will gather all the profitable experience and implement it right into your own website, that will allow you not to conduct your own tests saving your time and money and to avoid possible mistakes.
The truth. All the truth about doing this way we’ve already described in our previous publication named Why you shouldn’t trust A/B testing best practices. Here I’ll give a brief explanation why not to perform in such a way. Firstly, every business is unique with its own business niches, visitors, geography, and so on. That’s why in your own case you may get the completely different results than you saw in any respectable case study. That’s why the only proper way is to make your own analysis, to launch your own experiments and to get your own results which will be reliable only for your case.
Myth #8 – The Best Alternatives Always Look Nicer
The myth. The main sense of A/B testing is to find the best performing design and, it’s obvious, that the prettier design must be performing better. It’s impossible for an ugly alternative to win, so there is no point even to test such variations.
The truth. The truth is that the beauty of every single variation is only your own evaluative judgments. When one design seems to be better for one person, another person will choose another variant, because every man has own taste and vision. It may come to you as an unbelievable thing, but the point is to apply scientific approach instead of your subjective perception.
Myth #9 – Your Conversion Rate is So Good There Is No Chance to Improve It More
The myth. Your website already has a nice level of conversion rate, so you think there is no room for improvement and you’ve already reached the maximum of what is possible to get out of your website.
The truth. Even if your website already brings you enough money and you think there is no possibility to improve your conversion, there is no limit for perfection! There can be always some weaker areas in your conversion funnel. Anyway, if you’re about to start changing your website, A/B testing is the best methodology to do it!
Myth #10 – An A/B Test Without the Winner Variation is a Waste of Time
The myth. There are situations when you run an experiment and wait for the conclusive results for weeks and months. There is no clear winner because all of the variations show an almost similar conversion rate. So, in such cases, you’re just wasting your time and money implementing A/B tests!
The truth. There could be different causes for described situation. One of them is a little amount of traffic and another is that your changes are not really substantial for your website. In the first situation, it could take even years to finish the experiment (if you have, for example, 100 visitors per day). The second situation may occur when you’re focusing on incorrectly identified areas for improving. So, you should start from the scratch after rethinking what you are doing wrong in your A/B testing process. Our article about mistakes in A/B testing will be helpful in this case too.
