Google Website Optimizer and the iPhone...

What do they have in common? Even with the recent opening of Google Website Optimizer, much like the iPhone, you still have to hack GWO to get maximum value out of your testing. While I am grateful for improved support for factorial analyses and more help content, it would have been so easy to do better.

Here's the problem: Google Website Optimizer restricts your understanding of the effects of your experiment to a single outcome variable, like a conversion.

While getting consensus on a single overall evaluation criteria (OEC) is critical to successful ongoing testing and iteration in a business organization, you also want to use tests to improve the product teams understanding of the customer, products, the website experience, and their interactions.

Without the ability to go deeper to explain why an experimental condition drove the most conversions, you're simply playing roulette with your pixels, not building a better tuned product team.

So, yes, also like the iPhone, this limitation reduces the need for complex skills like statistical significance testing. (Not exposing a command line in the iPhone eliminates the challenge of unix).

There is a solution for those who aren't afraid of the truth... A set of enterprising analyst / coders have reverse engineered the GWO cookie and demonstrated how to port the values back over to Google Analytics. ROI Revolution shows how extract the GWO condition. While he shows integrating it with synthetic page tracker calls, I'd recommend using the "user defined" segmentation values via utmSetVar (old school) or pageTracker._setVar (new school ga.js).

Find more GWO power user tricks in my delicious feed gwo.

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Dan Thies's Gravatar Nice post, Andy! The first thought I had when I looked at WSO was "it only measures conversion??"
# Posted By Dan Thies | 4/16/08 4:32 PM
Built with BlogCFC, version 5.9. Contact Andy Edmonds or read more at Free IQ or SurfMind. © 2007.