Split testing, like any testing, can be enormously helpful in fine tuning your websites to be more effective sales channels for your business. The goal of the testing is to produce higher conversion rates, but I use the term conversion rate loosely here. For example, you can measure whether a user opts in into your mailing list, buys your product or clicks on your affiliate links. Split testing can be a powerful tool, however, like any other tool that produces hard measurements its results can be misinterpreted.

What is the purpose of split testing?

If you are selling an ebook on how to sell on ebay, what is the best headline to engage your visitor's interest? Is it "Make Money on eBay?" or "The Secret to Successful eBay Auctions?". Even if you are an ebay veteran, don't make the mistake of thinking you know the answer. You'll likely get it wrong. You will need to test different versions of your headline so your site visitors tell you the answer. I once wrote a headline that started off with "Who else wants to ...". I had seen similar long headlines on many other sales pages. My rationale was that if that style of headline was working on other sales pages, it should work on mine. Fortunately I decided to test it, and my statistics showed it performed poorly against my control headline! By listening to my site visitors through testing, I found out that they didn't like my "Who else" headline at all.

How does split testing work?

Here is how split testing (also known as A/B testing) works, half of your site visitors see one version of your site (the A version), the other half sees the another (the B version). You monitor the conversion rate of the two versions. The version that has the highest conversion rate is what you use going forward. It's often an iterative process, you keep testing changes until the conversion rates don't change very much from version to version. You are not limited to only testing the headline of your site. You can test other prominent components of your site that play a strategic part in converting your customers. For example, here are some website components you may consider testing: the main image of your landing page, the call to action, and even the promotional copy.

So when it is not useful?

Split testing is less useful when you change more than one component of your page at a time. If you change both the promotional copy and the main image on your B copy and it performs better, what does that tell you? That the image performed better or the copy? Sure you can use the B version of your site going forward, but what if the best combination was the A version of the image and the B version of of your sales copy? The only way to know is to test each change individually. If you want to test multiple versions of your site components, you should look into using a tool that supports multivariate testing. Multivariate testing is just what it sounds like, a way to test multiple changes at a time.

Another trap that testers fall into is putting too much significance on minor differences in the results. If you have 50 visitors to your site and 3 of them convert on the A copy and 4 of them convert on the B copy, is that justification enough to decide the B copy is the best version going forward? Not really. If you have a low traffic site, it's best to be patient and wait for more traffic, or put some effort into promoting the site to get more traffic so you have a more statistically significant test. Don't keep changing your site day by day trying to take advantage of what could be the result of a complete whim of your site visitor.