This article was written by Richard Mouser, testing genius and host of SplitTestJumpStart.com.
My First Split Test
In January 2006, my website www.MrWaterFilter.com was not such hot stuff. I was making sales, but found myself spending more on AdWords than I was making in profit.
I knew the website could do a lot better, but I wasn’t exactly sure how to get there. Some people were recommending that I add a “credibility seal” to my site – the “Hacker Safe” emblem – but I was reluctant to add the $1000+/year expense on top of an already in-the-red website.
What would you do, in my shoes?
A) Wimp out and make another change, hoping it will increase sales
B) Go for broke (not literally, let’s hope) with an expensive credibility seal
C) Sit paralyzed in the corner, praying for a miracle
No, no, none of those. You want another option:
D) Run a split test and let your visitors tell you if the credibility seal helps!
Lucky for me, I chose option "D." That first split test of the HackerSafe seal opened up a whole new world for me.
More on the results of that test later…
What’s a Split Test Anyway?
A split test is the simple, proven method to find out which of two web pages gets you more leads, opt-ins, sales or whatever you’re after.
Let’s back up a second. Every page on your site has a goal. With a given stream of traffic, each page guides a percentage of your visitors successfully to that goal.
When you run a split test, you simply alternate visitors between two different versions of a page. These two pages are your test pages.
You count visits to each test page and the number of visitors to each test page that reach your desired goal during the split test.
With those counts, it’s easy to determine which test page gets more visitors to your desired goal.
You know you have a winner when one page has a 97% statistical probability of getting visitors to your goal more often than the other page.
But you don’t have to worry about all the counting and statistics; there are free programs, like Google Website Optimizer that make the mechanics of split testing easy.
Best of all, you don’t have to be a technical genius to get split testing working on your website.
If you can fill out a web form, copy and paste text into a web page and upload files to your server, you can setup a Google Website Optimizer split test yourself.
If you don’t want to mess with any of the technology, you can even get your webmaster to do the techie part.
How does a Website Optimizer Split Test Work?
Let’s say you created a webpage named index.html. To run a Split Test with Website Optimizer, you just create another page with a different headline or a different picture, depending on exactly what you want to test.
Then you just upload that new page to your server with a different name, like indexb.html.
Next, you tell website optimizer the location of your two test pages and the location of the page that indicates a visitor has completed your desired action. This last page is usually a "Thank You" page.
Next, you copy some code from the Website Optimizer application and past it into your test pages and your "Thank You" page.
Finally, you upload the pages to your website and tell Website Optimizer to verify everything is ready to go. Then you just press a button to start the test.
That’s it; your Split Test is setup and running. See, the mechanics are easy.
Why Not Just Run Each Test Page for a Week and Compare the Results?
If split testing sounds complicated, this might be tempting. It sounds easier to just upload one version of the page this week and the other version next week.
You’d just compare results from the two weeks and know which one is better.
Don’t use this method. It doesn’t work.
Here’s the problem. There are hundreds of "environmental" variables that can affect your sales. Anything from the weather, to holidays, to major news stories, to celebrity endorsements, product reviews in the mass media, and the list goes on…
If one of those environmental variables changes, you’ll see a difference in sales. Most likely, you’ll attribute the change in sales to the page you’re monitoring, not to the environmental variable that actually caused the change.
So your results are wrong, and you have no way of knowing it. Using this method, you could be slowly killing your business.
You can verify this for yourself. Just look at your sales figures for any two weeks in recent history. Are they exactly the same? Not likely. Why was one week higher than the other?
There are a million environmental variables that could account for the change, but you’d never be able to identify them or adjust your results to account for them.
When running a test, you need to control as many environmental variables that could influence the results as possible.
A split test uses your current page as a control when evaluating the new page.
If something crazy happens during your split test, like, let’s say that Oprah happens to mention your website on national TV.
With a split test, both your original page and your new test page are subjected to the same crazy environmental changes Oprah would create.
This gives you a solid basis for comparing the results of the original page with the new page. This is why your original page is commonly called the control; it "controls" environmental factors.
When Should I Split Test My Web Pages?
Sorry, there’s no nice way to say this. Business is like war. On the internet, it’s a fast paced war. If you’re not already testing, you’re in danger.
You need to start testing now. You can stop testing as soon as you’re ready to let your competitors start slowly pushing you out of your market.
Someone out there is going after for your prospects right now. If they’re testing and you’re not, they’re slowly squeezing you out of your market…even if you don’t feel it yet.
You need to always test, so your site is always improving. Continuous testing is the path to survival, to winning, and finally to market domination (evil laugh).
You might eventually move on to apply your skills in a market with higher rewards, but once you really "get it", you’ll never stop testing.
But I Can’t Afford to Spend Time learning to Split Test…
Honestly, can you afford not to spend time developing your testing skills? What would a 50% increase in conversions to sale do for your business?
What would happen if your biggest competitor got that 50% increase instead of you? How would she use that extra profit to dominate your market?
If you’re the only person in your company, you’ve really got no choice. So just start working on learning to test bit by bit every day.
If you have a sharp marketer on your team, perhaps they could lead the test effort. But you’d best consider whether you should build testing skills yourself.
Yes, it takes effort to get started, but after that initial push, you’ll just have tests running all the time.
You’ll spend minimal time tracking the results and starting new tests as each one completes.
But I’m not Technical Enough to Do this…
Sometimes technical discussions around testing scare people away. Most of the time, these discussions are on very advanced topics that you don’t need to worry about.
The truth is, if you’ve got a website up and running, whether you did it yourself, or hired someone to build the site, you can do testing.
The work can be partitioned into marketing inputs to the test (done by you) and technical execution of the test (done by your webmaster).
In fact, Google Website Optimizer has a feature that sends detailed technical instructions to your webmaster for implementation.
You only need to focus on the marketing side of testing. Learn to split test so you can get to one other piece in the long run…
What Other Piece Do I Need to Learn?
Testing technology is available for free. It’s getting easier to use every day as more people use the tools and give feedback to the developers.
If the technology to test is available to everyone, you need to develop another marketing skill.
That skill is test design.
Testing tools are no different than any other kind of computer program. If you put garbage in, you get garbage out.
OK, in the case of test tools, you’ll at least get the better garbage out. But you get the point; it’s the inputs to the test tool — what you test — that really matter.
If you know what to test, how to create variations that get the results you want faster, and where to get a steady stream of testing ideas, you’ll be able to create and maintain a competitive advantage.
This is test design; it’s what testing is all about. With testing tools freely available, this is the next frontier where you can build a competitive advantage.
We’ve Covered a Lot of Ground, so here’s a Summary…
- Split Testing is an easy, proven way to determine which of two options performs best for any give page, or element on a page of your website.
- Free and easy to use tools like Google Website Optimizer open the world of testing to everyone.
- You need to start testing or you’ll be left behind as your competition starts testing.
- Running two different pages for one week each, then comparing results does not tell you which one performs better; you need to use split testing.
- You don’t have to be technical to test your landing pages, but you do have to put in some up front effort in to get started.
- Developing your test design skills is critical to long term online success.
What’s Your First Test Going to Give You?
That first test I described earlier gave me an amazing 76% increase in sales conversions.
From that moment on, I was hooked on testing.
With over 100 tests completed now, MrWaterFilter.com has gone from a marginal site to a cash cow. But it all started with that first test.
If you haven’t started testing your site yet, or if you’re just starting to test, I’ve created an in-depth series of articles and videos to give you a fast start with split testing.
- How to determine what to test first
- How to get started with Website Optimizer
- How to generate an ongoing stream of ideas for your tests
- How to develop your test design skills
All that and more is waiting for you at: http://www.splittestjumpstart.com.
Come on in and join the fun! Take that first step towards building your testing skills, increasing your profits and moving towards market domination.