Recently while doing some banner conversion rate testing with an eCommerce client, we found that our numbers were not adding up correctly. The problem was that Website Optimizer limits the test set to a specific action on the part of the consumer, essentially placing blinders on our testing efforts. With eCommerce sites though, there are many more variables to consider -- we want to know how many people clicked on the test as well as how many made a purchase.
Our solution was to add some new code that performs the logic to determine who see what, and let the custom segmentation within Google Analytics handle the responsibility of reporting the results. With this approach we're able to get a much more complete picture of any given test which allows us to more accurately determine ROI.
Sample Split Test
Let's go over a sample scenario before getting to the code. You run a site that sells headphones, and you're interested in testing a two versions of the homepage for a new type of high-end headphone. You're giving up prime real-estate by running this split test on your homepage, so it's important to earn as much as you can.
Our simplified consumer experience might look a bit like this:
If we're only using Website Optimizer the issue then becomes, from which point to which point do we consider to be a conversion? In Website Optimizer we can only select a single point A and a single point B, we can't track the customer's path through the entire funnel. That's the unacceptable dilemma we're working to overcome. If we have single point-to-point tracking then we can only pick one of the following items to measure:
- Are visitors clicking on the banners?
- Are visitors adding the product to the shopping cart?
- Are visitors making the purchase?
None of these individual conversion rates can tell the entire story -- we need more information.
Getting Our Hands Dirty
Website Optimizer is a great tool to track any one of these paths, but it cannot track all of them the way Google Analytics can. Our goal then, is to create an easy to use system that will allow us to track all of these paths and help us identify where we're losing customers.
I'm including the PHP code we used for those of you that may find it useful. If coding is not something you enjoy then the PHP code below may look daunting. Conceptually though, this is all it's doing:
We want to make sure that if a user comes back while the split test is still going on, that they have the same experience.
You would call this code with the following:
Additionally, while you're testing the layouts, you don't want your team to fall into the same segments while they're creating your beautiful new homepage. To avoid this, you can use the override call that's built into the class:
That way you and your team can always switch back and forth between the various layouts.
One thing to know about custom segmentation, Google Analytics only allows users to be grouped into a single segment. If your site is already using segments for another internal process, chances are that this approach will not work for your team. This also means that you can only run a single test at a time, but that's not really a negative. You're only running one split test at a time, right?
This approach may be over-simplified for your needs, but it can go a long way towards getting your team all the information it needs to make great decisions.