AB Testing - e-commerce

The Net Effect: Eyes That Buy

Wed, 03 Sep 2008 13:28:00 -0700

At StomperNet, where I serve as staff and faculty, we're giving away Stomping the Search Engines 2, a rewrite of the original empire launching STSE from Brad Fallon and Andy Jenkins.

Shown at right is Page 2 of Eyes That Buy from The Net EffectVolume 1, Issue 1

Check out the video for some of StomperNet's success stories -- it's amazing. These guys launched their internet business training program after creating successful online e-commerce sites, and have become a juggernaut of internet business success -- see the StomperNet Universe of freely available content.

So you can get an 8 hour DVD course if you take us up on the ask to you try a subscription to our new print publication, "The Net Effect" (and pay shipping & handling).

Here's a snapshot of a moment from the video where my article in the first issue of the Net Effect is featured:

The first page is full of an explanation of how humans interact through vision with web pages, while page 2 focuses on take-aways. You see 3 of about 7 tips here.

Use color smartly. A single item varying in color is processed by hardware early in the processing flow, or pre-attentively, and creates visual popout, drawing the eye.
Chunk with implied boundaries: Forget horizontal rule (HR) tags, they're useless to the peripheral vision. Use implied borders and Gestalt principles visual completion.
Break the Grid: Symmetry is pleasing to humans, and a well balanced page communicates ease of use to the user, but you can break this symmetry very strategically to call attention to a location.

So, you can get the first issue and a full course on SEO from some of the best over on this salesletter. (Yes, somehow the sales letter genre manages to break many of the rules of web design, but it is chunked!)

Product Video Boosts Conversion Dramatically at MyWeddingFavors.com

Fri, 01 Aug 2008 11:36:00 -0700

Internet Retailer is running a story based upon a case study from our video hosting & creation vendor SilverDock. This result is actually several months old and was included in Marketing "Sherpa's 2008 Wisdom Report":

We tested product videos on our top 10 revenue generating products. After several rounds of iteration improving the load experience, we settled on a 3 level test: no video, autoplay, and click to play. Both video conditions resulted in conversion gains, but the click to play was dramatically superior. In addition to a 35% increase in conversion for the products with videos, the transaction funnel became much straighter. In other words, users who chose to click to play a video and then added it to cart were much more likely to complete the checkout.

Wedding related shopping is kind of a bellwether for purchasing processes that have to be perfect and are conducted socially. Product videos enhanced the sales of our best product spectacularly, reducing the chance of erroneous add to carts and increasing buyer motivation. Time to go put some video on your site!

StomperNet: Going Natural 3 - Adwords Triangulation Method

Tue, 06 May 2008 18:15:00 -0700

We launched the Stomper Scrutinizer within the video series called Going Natural 2 back in December, and now it's time for GN3. The first video in the series showcases Dan Thies talking about his strategies for AdWords. Most notable is his description of how to do split testing with AdWords for fractions of traffic. In essence, if you want to dedicate 10% of traffic to a new ad, you create 10 variants of which 9 are identical. This also provides an "A-A" test that can help you understand variability in your data. Watch the video. There's a bonus video, new excerpts from the Going Natural 1 Series, along with a downloadable version of my video on understanding vision and web design called Click Fu.

Google Analytic Gems #2: Quantifying Deliberative Conversions

Tue, 08 Apr 2008 06:42:00 -0700

Another little known gem in Google Analytics is the Time to Purchase and Visits to Purchase.

For MyWeddingFavors, where the purchase is an exceptionally meaningful one for our customers, some 40% of our sales happen on subsequent visits:
. Careful though! Looking into Days to Purchase, we see only 25% of sales happen on a different day than the introduction to the customer.

So many (15%) of our shopping experiences are stretched out over a day, while only 25% happen on a subsequent day visit. Understanding this pattern has some serious implications for design, business strategy, and e-commerce feature set.

Have you authored a CLICK HERE link lately?

Wed, 23 Jan 2008 07:25:00 -0700

Tsk tsk. "Click here" is one of the most extensively used bad interface elements on the web. Descriptive hyperlinks are way more effective -- a drum that industry gurus have been arguing for some time. There's some new data from Internet-Based Research group at UWash.

Spyridakis, J.H., Mobrand, K.A., Cuddihy, E.,and C.Y. Wei. Using Structural Cues to Guide Readers on the Internet. Information Design Journal, (in press).

Users got more out of the content in the Sem/Org link configuration than any other. Sem was the next runner up.

This illustrates the value of descriptive, informative links... not simply mechanical expressions like "next" or "click here".

One productive way to think about this is to imagine that the links the user clicks on are part of a conversation. It's quite boring to talk about the mechanics of using your web browser. If you want to impact the user in a significant way, whether it be to inform or sell, writing descriptive and informative links is the way to go.

If you're not convinced, check out some other views.

A taxonomy of motivations for website testing (A/B split, multivariate)

Fri, 28 Dec 2007 08:06:00 -0700

The intense competition for business on the internet has created an environment in which user interface experimentation is a critical process that can provide exceptional return on investment.

Here's my take on an inventory of ways to apply testing:

Business acceptance testing
Value estimation
Design choice determination
Customer understanding

Business Acceptance Testing

Example: You want to add yet another shortcut on the homepage for a new sub-audience. Use testing to validate you didn't mess up the other functions of the homepage.

Value Estimation

That new search function is going to cost you X thousands of dollars. How long will it take to provide a positive return on investment.

Design Choice Determination

The boss thinks the logo should be purple. You don't.

Customer Understanding

Multivariate methods are really valuable for this use case. Say you wanted to ask the question, "Is copy at the top of our product pages worthwhile? Or should we just drop it and get more products above the fold?"
A multivariate test can let you vary the size & quality of the copy, along with other elements that push the product down the page, and assess the general impact of products higher on the page as well as the general impact of good copy.

Scrutinizing the Shopping Experience

Wed, 26 Dec 2007 13:26:00 -0700

Over at StomperNet, we've just released the Scrutinizer browser as a free download.

Here's a (video only) screencast of me using the Scrutinzer browser to shop for the XBOX 360 game "Assassin's Creed" at Amazon. It takes me 2 minutes. First, I find the 360 games category quickly @0:20, but chose the wrong game genre category, fighting @0:43. Next up, I chose adventure @1:08 (turns out it's in action).

After two wrong tries, I resort to search @1:15, but I'm set to search in the adventure category and have to reset it to search all xbox games. I finally reach the page @1:38, and add-to-cart, success.

We produced a super engaging 30 minute video with animation explaining this, but in short, the tool simulates the limited high resolution vision available to humans as they use the computer. The visual simulation allows me to zoom in and out of the page and illustrates the tunnel vision a user has given a specific goal.

The tool is freely available (happy holidays!) and you can learn more about the principles behind it, how to use it, and get the download at http://about.stompernet.com/scrutinizer. Shout out to the WebMetrics Guru blog who garnered some interesting insights from the video alone, and to The Scobleizer for the generous props.

Google Analytics Site Search: Usability & Business Goals

Sat, 10 Nov 2007 10:42:00 -0700

So Google Analytics site search features have been out for a week. Here's a unique insight that can be garnered by studying search queries from a key location, the shopping cart page, in an e-commerce site. This collage shows the a drilldown view from Content -> Site Search -> Start Pages to the search terms that were issued from our add to cart page in the last week. We see "invitations" and "camera" where it seems the wedding participant to be is trying to knock out a few more items from the todo list. While upsells can be done poorly, as described in Top SURL's 10 Shopping Cart Design Errors, good web site design is not just about usability. It's about maximizing the intersection of the user goals and the business goals. If some users are slightly distracted by an upsell that a smaller number find appealing, and that contributes to the bottom line (e.g. it's not obnoxious enough to cause users to bail on the process), that's solid user experience design. SURL cautions against forced upsell "interstituals" during the add to cart process. Best practice is to display these upsells on the cart result page, not as a forced interruption during the process. The new site search features have already contributed to the bottom line in our e-commerce business. My full analysis, training, and case studies are available to members of StomperNet but I'll be sharing some of the more unique insights here in the future.

Stretching My Operating Framework: Association of Consumer Research

Thu, 25 Oct 2007 06:46:00 -0700

I'm off to attend the ACR conference in Memphis this weekend, participating in Roundtable Session: Internet Tracking and Clickstream Data: Methodological Issues. Some of the talks I'm looking forward to:

Scale Development and Measurement Issues
Consumer Response to Aesthetic Aspects of Product Design: 1-, 2-, and 3-Dimensional Perspectives
Social Contagion Effects in Marketing
Watching Gen Y Grow Up: Consumer Behavior Across Lifespans
What Drives Word of Mouth: A Multi-Disciplinary Perspective

This genre of research investigation is a bit far from my roots in cognitive science / psychology, but I'm sure it will be enlightening. Stay tuned for the insights.

Q&A: Can I test with only 5000 visitors a month?

Wed, 24 Oct 2007 07:28:00 -0700

Five thousand visitors a month is not too little traffic to effectively test. Here's a nice writeup of small sample sizes and simple statistical testing on conversion rates. If you site does not have transactional components (low conversion, high gain), then it may be too little traffic to measure more highly variable dependent variables like bounce rate, depth of visit, or more refined engagement metrics. As long as you have a contact, purchase, or download outcome, the techniques illustrated in the AdWords stats post will work for you. Multivariate testing actually eeks more statistical power out of less traffic than simple split testing; it's just that the conclusion of a single "test" takes longer, but you get more done in a year than doing iterative splits. I've worked in situations of immense scale -- you still have to test for a full week to deal with day of week effects. For the low traffic scenario, if you focus on key points along the line to conversion, you *can* do data driven design iteration. The Luna Metrics Blog covers this same topic, with details of the mailing list post that triggered it. For those with the opposite scenario, lots of traffic, check a video excerpt from my web 2.0 expo talk on analytics at scale.

Interview on Search Behavior with Jim Jansen: Query Length and More

Sat, 28 Jul 2007 06:40:00 -0700

I'm just back from SIGIR 2007 where I interviewed Jim Jansen on his long term research on query strings as well as more recent work on the nature (quality, perception) of search engine sponsored ads. Finally, we talk about new work Jim presented at SIGIR studying the search process as learning (not decision making).

Check it out at Free IQ.

Panel at the Eight World Congress on E-Business Management

Mon, 25 Jun 2007 05:22:00 -0700

I'll be in Toronto in a few weeks, speaking at the E-Business Congress in a panel on eManagement in the Dynamic Digital Environment.

This panel combines individual presentations with a group discussion. The focus of this session and the expertise of this panel bring together three different facets for managing in a Digital, eBusiness environment. 1) the evolving relationships among managers and their employees; 2) a leadership and advocacy approach to managing digital creation and collaboration; and 3) a performance-based management approach for eCommerce-related work.

Naturally, I'll be focusing on #3, but great strategies for management of collaboration in metric driven management is also very interesting to reflect upon.

Luckily, it's prime patio seasion in Toronto and I'll be there for a test run of a local "Web Analytics Wednesday".

What's a good conversion rate?

Mon, 11 Jun 2007 09:20:00 -0700

From the MineThatData blog comes a nice analysis of Internet Retailer Top 500 e-commerce stores conversion rates.

Catalogers have the highest website conversion rate at 4.89%, followed by Web-Only businesses at 3.92%, Retail Chains at 3.05% and Consumer Brand Manufacturers at 2.99%.

Of course, any professional will tell you comparing conversion rates is one of the sillier ways to waste time in analytics. Traffic generation techniques are a key reason why converion varies. You might be fighting for traffic in a very competitive area, grabbing cheap volume terms tangential to your pitch and long tail keywords, or you might be trying to max out your sales volume given a solid investment in inventory. The quality of the traffic is a very strong differentiator.

Still, what's a good conversion rate is a question I receive all the time, so having some "big company" metrics is a nice contribution to the zeitgeist. In addition, this range of conversion jives pretty well with my experience. Branded terms can easily have conversion rates at 5% given that the users are either returning visitors or otherwise referrals.

Term Alert: Pre-Bounce

Thu, 07 Jun 2007 20:11:00 -0700

While bounce rates are not enemy #1 for everyone, the transition in your conversion funnel which loses the highest percentage of users is the biggest issue -- typically abandoned carts for e-commerce shops, I really enjoyed this post coining the term pre-bounce.

The Pre-Bounce is a user rejecting your site based upon visual previews popping up all over the web. Aesthetics matter.

E-commerce Tips: Coupon Codes

Sun, 13 May 2007 09:01:00 -0700

Steven Spencer has a nice series of E-Commerce Tips. There's not much to argue with here, but I found the ideas on coupon codes especially interesting. Consumers reactions to paying more than they have to, implied by the presence of a coupon code form element in the checkout process, is often very strong.