Don't Play Roulette with your Split Testing Efforts

While split or multivariate testing is an extremely powerful tool, it should not be deployed as the sole method driving improvements to your online business. If testing is applied as the only way to determine the right path forward the outcome is certain to be less successful over the long term.

At the usability professional's association a few weeks ago in Portland, I chatted with Beverly Freeman @ Ebay and she recorded the essence on the whiteboard:

My premise is that testing is much more productive when it is aimed at hypothesis testing because the cumuluative outcomes of multiple tests create a body of knowledge about your business, your users, and their interaction.

The next point in the white board is about triangulation. Analytics can provide strong statistically meaningful results, but making sure that rationalize a "why" is important.

The general pattern here is that you can identify in the usability lab or other high resolution, low volume observation scenario and quantify with testing. Alternatively, you may observe in analytics and engage with customers in lab testing or other inquiry to understand.

Comments (Comment Moderation is enabled. Your comment will not appear until approved.)
Tammy's Gravatar I completely agree! I have had this argument with clients, who don't understand that if you don't test it- you don't know what is working! Thank you for this post.
# Posted By Tammy | 8/19/09 5:01 PM
Built with BlogCFC, version 5.9. Contact Andy Edmonds or read more at Free IQ or SurfMind. © 2007.