A taxonomy of motivations for website testing (A/B split, multivariate)

The intense competition for business on the internet has created an environment in which user interface experimentation is a critical process that can provide exceptional return on investment.

Here's my take on an inventory of ways to apply testing:

• Business acceptance testing
• Value estimation
• Design choice determination
• Customer understanding

Business Acceptance Testing

Example: You want to add yet another shortcut on the homepage for a new sub-audience. Use testing to validate you didn't mess up the other functions of the homepage.

Value Estimation

That new search function is going to cost you X thousands of dollars. How long will it take to provide a positive return on investment.

Design Choice Determination

The boss thinks the logo should be purple. You don't.

Customer Understanding

Multivariate methods are really valuable for this use case. Say you wanted to ask the question, "Is copy at the top of our product pages worthwhile? Or should we just drop it and get more products above the fold?"
A multivariate test can let you vary the size & quality of the copy, along with other elements that push the product down the page, and assess the general impact of products higher on the page as well as the general impact of good copy.