Introduction

Interested in tuning your e-commerce site to increase your conversion rate?

In this article, we’ll present steps to help your site sell more. Large online retailers seeking to improve their sites can afford substantial investments in sophisticated analysis tools and costly usability consulting. Smaller retailers often can’t.

This article aims to help folks seeking to tune sites “on the cheap.” Read on for low-cost high-impact ideas on increasing your site conversion.

List, Offer, Package

Let’s start with expectation management.

There’s an old rule of direct mail that says the three key elements for success are, in descending importance, “list,” “offer,” and “package.” “List” means having a large universe of high-quality prospects. “Offer” means having a good selection of relevant merchandise, fairly priced, with a solid value proposition. And “package” means a clear and compelling marketing vehicle, with strong copy and design.

While all three elements are important, “offer” trumps “package” and “list” trumps “offer.”

That is, if you want to double your web sales, the fastest and most cost-effective way to do it is to double the number of visits from qualified prospects. Your next best option would be improving your offer: broadening your selection, sharpening your prices, and improving your value proposition. Optimizing the “package” - that is, the site organization, layout, and usability - comes in third.

So should you worry about conversion at all, or just focus on driving more visits? Both are essential. Even coming in third, how your site looks and operates is important. Tuning the site helps you extract full value from all your hard-won inbound traffic.

Disclaimer finished, we begin.

Method

Here’s a generic four step process for tuning your site.

1. Determine what on the site might need improvement.
2. Come up with an alternative page or pages making that improvement.
3. Run an A/B test to compare the alternative to the control. If the alternative is significantly better, roll it out.
4. Repeat.

Step 1: What on your site needs improvement?

You may already know some of the trouble spots on your web site after listening to comments from customers and employees. You may also note weaknesses on your site after surfing and shopping from your competition. And recall Neilsen’s “Law of Web Usability”, which states your visitors spend most of their web time not on your site - pay special attention where your site uses atypical navigation or layout.

As a rule of thumb, the four most important components of an e-commerce site are the product page, the site search, the cart, and the home page.

Running “discount usability sessions” is an inexpensive approach for finding possible improvements to the important components of your site.

Discount Web Usability Testing

“Discount usability engineering” (a phrase coined by web guru Jakob Neilsen) is a great way to determine what on your site trips up your users. You can read more on this method at Neilsen’s site, useit.com. Here’s a quick overview.

Recruit five local web users who are not familiar with your site. Pick folks who resemble your target demographic. Schedule five 45 minute sessions on the hour, one for each user. Offer each user a modest thank-you gift for participating: a gift certificate or cash. Pick a friendly person in your organization who’s comfortable with the web to serve as your moderator. Don’t pick someone who works on your site. Prepare a quiet private room with a computer and a video-camera. Have the moderator welcome each user and introduce the exercise:

“Welcome, Mary, thanks for coming in. My name is Lisa. As I described on the phone, we’re watching people use our site so we can try to make it better. I don’t work on the site myself, so don’t feel bad saying if you find something that’s confusing. All your comments, good and bad, are helpful to us. If OK with you, I’ll videotape our session; that will help me later with my notes. This session will take us no more than 45 minutes.”

The moderator then asks the user to complete a task on the site, sharing their thoughts as they go.

“Ready, Mary? Great! I’d like you to shop for and purchase a gift for an out-of-town friend using our website. You’d like to spend around $75 on the gift. As you use the site, if you could share what you’re thinking and expecting, that’d be great. Before clicking a link or pushing a button, please tell me out loud what you expect to happen next - what sort of page will come up, what you’ll see, etc. Make sense? Excellent! So let’s begin. What are you thinking when you see the homepage? Where would you click first? Why? And what do you think will happen after you click? ”

The moderator should take notes, detailing where users stumbled and where the site didn’t do what was expected. The moderator will need to encourage users to keep sharing their thoughts; but the moderator shouldn’t lead the users.

The goal of discount usability sessions isn’t quantitative statistics - it makes little sense to compute statistics on a sample of five. Rather, you’re seeking trends and “Aha!” insights. The insights needn’t be earthshaking, just places where many of your users hesitate or stumble.

Here are three examples of modest “aha!” insights seen in different usability sessions:

• On the “view cart page”, below the table showing cart contents, there were three buttons. The buttons read, from left to right, “Checkout”, “Continue Shopping” and “Empty Cart”. In testing, multiple users instinctively hit the rightmost button to continue, unintentionally purging their cart.
• The product hierarchy included “Sale Specials” as a product category. As a result, once a certain dress or top was placed in the sale area, it no longer showed up under the “Women’s” category. Users expected to find sale items alongside the other dresses, and thus couldn’t find them.
• The product page displayed a grid of color swatches below the large item image. When there were more than three color options, the grid pushed the copy block, the price, and availability “below the fold” on a standard browser.

Advice: when running usability sessions, don’t use your achingly fast T3 connection and your web designer’s gigantic display. To mimic reality, users should experience your site on a dialup (or slow DSL) using a regular size screen.

Step 2: A better alternative

Once you’ve identified a possible site problem, the potential solution is often clear. (”Remove the ‘Empty Cart’ button altogether, move ‘Checkout’ to the right-most spot and make ‘Checkout’ more prominent.” “Make sure sale items also display in their original category.” “Move the copy block up to the right of the image and move the grid below the image, so price and copy are always visible without scrolling.”)

Testing is expensive, so make sure only to testing changes significant enough to matter. Testing whether your navigation should be light blue or light green probably isn’t worth it. Testing a single large “hero” product homepage treatment versus a top-selling product grid treatment probably is worth it.

Step 3: Running an A/B test via the “Day Swapping”

Is your new idea better than what you’re currently running? Only testing can say for sure.

Larger retailers can build or buy expensive software to run multiple versions of the site simultaneously. Such packages randomly assign visitors to the control or the alternative treatment, ensure a visitor experiences the same treatment consistently throughout their visit (and on subsequent visits), accumulate data for analysis, and report on results.

With a little work, a smaller retailer can gain much of this benefit without needing the costly software via a 7 day “day swapping” test. Here’s how.

Build two versions of your page in different files. For example, for testing your homepage, you would create “index-control.html” (a straight copy of your current homepage, “index.html”) and “index-alternative.html” (a new page implementing your new idea.) For testing design and layout on a CSS site you would make two versions of your stylesheet: “style-control.css” (a copy) and “style-alternative.css” (new). For testing the cart, you would make two versions of the cart script: “checkout-control.asp” and “checkout-alternative.asp.”

Create simple scripts to copy one of the versions onto the live page. For example, if you were doing a home page test on a Unix or Linux platform, these two trivial bash scripts would perform the copy (after adjusting filenames to match your server configuration):

make-control-active:

#/bin/bash
# script one: make the control active
cp /var/www/public/index-control.html /var/www/public/index.html

make-alternative-active:

#/bin/bash
# script two: make the alternative active
cp /var/www/public/index-alternative.html /var/www/public/index.html

Monday morning at 9am, run the “make-control-active” script. Tuesday morning at 9am, run the “make-alternative-active” script. Repeat this process for two weeks, alternating each day. (You can automate this with a scheduler like “cron” or “at”; otherwise, bribe your system administrator with donuts.)

Analyzing A Day-Swapping Test

At the end the 14 days, collect daily metrics for the period: site sales by day, average pages-viewed-by-visitor by day, etc. Create a spreadsheet for each metric by day of week, noting how many times the alternative beats the control. Here’s an example:

	Site Sales
	Control	Alternative	Alternative better?
Monday (June 6 & 13)	$ 4,384	$ 4,514	YES
Tuesday (June 7 & 14)	$ 4,453	$ 4,713	YES
Wednesday (June 8 & 15)	$ 4,315	$ 4,640	YES
Thursday (June 9 & 16)	$ 4,536	$ 4,866	YES
Friday (June 10 & 17)	$ 4,577	$ 4,547	no
Saturday (June 11 & 18)	$ 3,900	$ 4,197	YES
Sunday (June 12 & 19)	$ 3,719	$ 3,676	no

As a rule of thumb, if the alternative beats the control 6 or 7 days out of the 7 day-pairs, it is significantly better. If the control beats the alternative 6 or 7 days out of the week, the alternative is significantly worse. (For stats enthusiasts, this is a non-parametric sign test with alpha at 15%).

This simple design sidesteps day-of-week effects (due to the matched design), season sales trends (due to the alternation), and outliers (the non-parametric test doesn’t place undue emphasis on the occasional huge order). This design works best for retailers who close a large fraction of their sales on a customer’s first visit, rather than for retailers who typical experience long delays between visit and order.

For more statistical power, you could run the test longer. If you ran the test for four weeks, you would still alternate day by day (more donuts!), comparing the first Monday to the second Monday, the third Monday to the fourth Monday, and so on. After four weeks, you would claim the alternative the winner if it beat the control on 12, 13, or 14 of the 14 day-pairs, and claim the alternative a loser if it beat the control only 0, 1, or 2 days of the 14.

You might feel these cut-offs are too rigorous, and that it would be very hard to find a new alternative that performed so strongly.

Yes. It will be hard. You will find that many of your tests yield indifferent results, with the alternative neither provably worse nor provably better than the control. Sadly, that’s the reality of trying to better a reasonably good website.

Expect indifferent results 4 out 5 times. Have patience and keep on trying - have faith that through smart ongoing testing, you will eventually “crack the code” and find something that really does improve your site conversion.

Since finding changes that yield statically significant sales increases is challenging, consider proxy metrics as well. Suppose your alternate design did not provably increase site conversion (but also did not provable harm it), but did provably increase average time spent on site and average number of pages viewed. That’s compelling evidence to roll-out the change.

Conclusion

You work hard to get visitors to your site. Using the one-two punch of discount usability sessions followed by A/B testing, with some luck and a good deal of patience you can increase site conversion. Now’s the ideal time to start, so you can have your site in better shape for the holidays. Good luck!