How to create the best A/B testing strategy

I received a great question today about how to best set up A/B testing. I’ll try and shed some light on how I approach A/B testing for both products and services.

The question was: "Does A/B testing cost money and how do I create the best A/B testing strategy?"

My reply:
A/B testing does cost money but the point of A/B testing is to make money from it. When I worked at ACDSee.com we did over 1 million downloads per month and my job was to convert free users into becoming paid users. I grew my channel by 13% year over year. How? A/B testing. I’ll tell you how.

With a team of developers, designers, copy writers etc…same team as you listed on your blog, we were still profitable because we made A/B testing our mantra.

Here is how I managed 6 software products in 12 languages.

Metrics and templates. I had the developers build very sophisticated tracking software that I can’t disclose here. This software was so smart that I knew everything about a user. When they clicked, bought, location even what they had for dinner. (wink wink). I then developed templates that I knew was converting at xx%.

My mission was then to try to beat these templates by not using designers and programmers. I did this by having systems and processes in place where I could manage testing environments myself. That way I had high costs upfront in building, but when the actual test started I managed it myself via the system I built.

I knew that “landing page 1” performed at xx% but LP 3 performed at a higher % so looking at traffic sources, where in the buying cycle they were I could change the messaging on the spot to boost conversion rates.

This is hard to explain but think of my approach as Lego or building blocks. I could easily replace one “bad block” with a “block” that I knew was performing, thus boosting conversion rates.

A/B testing is all how you set it up and track progress. Word of caution though, if you start with good material it will make your AB Tests a lot easier because you don’t have to re-develop designs, programming etc. Do it right the first time and the rest will be easier to manage.

For all of you doing A/B Testing I hope this will help you!

Chris