We recently had the enjoyable opportunity to interview web analytics guru Avinash Kaushik.
Hi, Avinash! Thanks for the interview today. For readers not familiar with your work, you write an excellent blog about web analytics called Occam's Razor.
Can you tell us about the history of your blog -- why you started it, what you've learned from blogging, and where you see the site heading?
I had been presenting at conferences etc and some people would always say “hey you should have a blog”. Finally at a event in Jan 2006 Andy Beal finally convinced me that I really need a blog to change my life! I spent around three odd months studying the blogging world, following some interesting blogs and participating in the blogosphere to get a feel for thing. I launched Occam’s Razor on May 16th 2006.
I am absolutely stunned at how many people read the blog and participate in the conversation. Last month [Feb 2007] the blog received around 19k Visits from 11k Unique Visitors. That is completely unanticipated, more so because unlike most blogs I only write around four to six times a month now (used to be a bit more).
Through their comments readers of the blog have contributed just as much content as I have written (around 150k words), I am very grateful for that. You can imagine very easily how that makes for a wonderfully stream of great education for me (and other readers).
The core purpose of the site will stay unchanged as long as I can help it: to educate folks on the topic of web research and analytics and to put forth a alternative point of view to what else is out there.
Until recently, you were at Intuit, and now you've started your own firm. Can you tell us more about your background and where you're heading next?
I am a Independent Consultant now, my company is called ZQ Insights. My hope is to do value added consulting on all things related to making decisions on the web. My ideal client is one that wants to turbocharge their Web Analytics efforts or create truly data driven organizations and does not know how to do it.
Clickstream analysis, Outcomes measurement, Web Research, Experimentation and Testing, Competitive Intelligence and more... I hope to help organizations large and small unleash the power that is inherent in this wonderful world we call the Web.
My first client is Google, I am currently the Analytics Evangelist for Google Marketing.
I am also looking forward to my book coming out next month (Web Analytics: An Hour a Day), should be a exciting few months.
Turning now to web analytics: in your opinion, how big an issue is web analytics?
For a typical retail site, what sorts of sales increases might a retailer expect to see from a strong analytics program?
And what sort of sales increases might a "home run" web analytics insight generate?
It is my humble opinion is that the Web is perhaps the cheapest and most effective channel on the planet right now, and it does not matter if you use it to generate sales or to do customer support or service or simply to convince the world of your greatness. No matter what you do you can do it cheaper, faster and more efficiently on the Web.
If you buy that, even a little bit, then the answer to your question is: There are no limits. The only limits are imposed by your ability to apply the right mindset, process and resources. Period.
With application of basic analytics and even a part time analyst I have seen increases in sales directly attributed to insights from web analytics of twenty five to one hundred percent of the investment in the tools (remember you can start free) and resources.
There is no upper limit really. Consider this, do nothing else except apply slightly robust analytics to your PPC / SEM (Search Engine Marketing) campaigns. Your ROI? I guarantee you that you’ll get twenty five percent on the first day.
Like JFK said, “Think not what Web Analytics can do for you, think what you can invest in Web Analytics, and ROI will follow.”
I am sure I have butchered that quote!
One of my favorite Kaushik quotes is your "10/90" rule, which says for sites with $100k to spend on analytics, they should invest $10k in software and invest $90k in great staff. Can you expand on the 10/90 thinking for us? Why doesn't or why can't the fancy web analytics tools produce million dollar insights at a touch of a button?
The rule comes from my own prior experience as well as my extensive collaborations with Fortune 100 and Fortune One Million companies and from doing root cause analysis of why most of them find themselves in a soup when it comes to web analytics.
The Web is inherently complex, every bit of it. Pages, urls, search engines, customer needs and wants, customer behavior, primary purpose of each customer segment, data capture and all the problems associated with it. Then consider that businesses are inherently complex. Multiple channels, complex marketing campaigns, divisional silos, existing processes, or lack there of, relationships between changes to the site and outcomes etc.
Unlike any other channel this one has the least rationality around it, not matter how much you wish that it did. And it changes every day.
In such a environment insights come not from the multi million dollar tools that you can implement, and yes you can buy multi million dollar tools, but the human power you can unleash to make sense of all the irrationality and ensure that valuable nuggets of insights can be found.
Tool are great a logic and structure, and we have been OK in traditional decision making worlds (think ERP, CRM, etc). But the Web needs a different approach. It is the strong interpretive layer of ambiguous “artificial” intelligence there is required to make sense of it all.
One day things will get stable and sensible enough that you deploy expensive tools and on Day One simply because they are sexy that they will produce automated insights. Until 2025 rolls around we are all better off investing in the raw human brain power you need to apply today to get the insights that will “move the dial” for your business.
Let me stress that the answer is not that everyone should cancel their contract and implement a free tool. The answer is that the tool is not the answer, it’s the people. Buy the tool you want, but remember the 10/90 rule and invest accordingly if you want to win.
You've also suggested a 80/20 rule for reporting: analysts should spend 80% of their time doing analysis and only 20% reporting. How much data should a good web analytics dashboard contain? How many key metrics should an online retailer track to understand their site? And which ones?
The stress on analysis is because with reporting all you are doing is throwing data out there, usually highly aggregated or deeply detailed, both good exercises to flex data flow muscles but rarely would they make you the lean mean fighting machine that you want to be. Analysis is the art of probing data, of discovering trends, digging beneath the surface, marrying the improbables and in the end finding insights. That’s how you make money.
Take any organization with remote success with analytics, none of them got there by having robust reporting programs. They got there by the sheer courage, hard work of solid analysts who took time to dig and probe and recommend.
I think I am on the record on the blog saying that you should only have eight to ten metrics on a dashboard, that you should be able to print it on one page, in eleven size font or greater. For more please see the post: Six rules for actionable dashboards.
There is no pat answer how many metrics each retailer should track. It depends. That’s not a cop out, rather it is a reflection at how there is little in common in any business. Two retailers, say Best Buy and Circuit City, can have radically different strategies to leverage the web and their analytics strategy will have to fit around their unique web strategy. For example Circuit City will give you a $24 gift card if your online order is not reader for you to pick up in the store in 24 minutes. Best Buy will do no such thing. Tiny example of how your web analytics will be different in each case.
OK, there are some metrics I love and adore that everyone should measure. On the web we all do a very poor job of understanding the customer needs and wants and thus their experience on our sites. ClickStream data is pretty sub optimal at representing customer experience, so using qualitative methodologies I am a fan of measuring Customer Satisfaction (“were you satisfied with your experience on our site today”), Primary Purpose (“why are you here today”) and Task Completion Rate (“were you able to complete the task today”).
Measure those three using Surveys on your site, you can do Site visits with your customers, you can do remote usability studies. You will find them to be a great complement to your web analytics clickstream data.
Really interesting! Most folks, myself included, immediately think of clickstream stats when we hear says "web analytics." Your recommendation to use site surveys to track customer satisfaction and task completion -- that is, listening to customers, before diving into the logfiles -- is comforting and wise advice.
OK, moving from web analytics over to prospective testing. Do you advocate testing many site changes at once (multivariate testing, Taguchi, etc), or rather single element changes sequentially (A/B, n-th tests, split tests, etc)?
You should test as much as your intelligence will allow you to.
Testing is exercise that requires a non-trivial amount of expertise to perform at a sustained level. You need cross functional knowledge, deep knowledge of your customers, a solid grasp of data, lots of outside the box creativity and of course someone who can help you create relevant tests.
With that said you should do as many tests of as many types as you think put good ideas out there.
I recommend you can start with simple A/B, migrate to complex Multivariate, and end up with “Experience” tests that test customer experience and concepts that span multiple pages.
If you want to have life-altering web site gains, remember, those won’t happen because you have improved one page on your site or a set of single pages in a silo. Life would be good if customers were that easily convinced! You need bigger changes to get bigger results.
Beyond '90/10' and '80/20', what are the three most important things a large retailer should be doing to get the most of their web analytics effort?
- Don’t focus solely on improving conversion rate, if you can.
- See the last answer, testing is God’s gift to you and your customers, please consider investing in it.
- Empower your analysts, let them find the insights, set them up for success to do the kinds of things outlined above (the % schedule).
Interesting! How about a small retailer? Same tips, or different three?
In my humble opinion smaller retailers are more at a advantage on the web because they are inherently agile and can think smart and move fast. They don’t quite have the budgets to buy expensive tools (clickstream or testing etc) but they don’t suffer from bureaucracy and needing to satisfy lots of reporting for lots of stake holders.
My advice to them is to invest in time and resources to analyze the data and they’ll be surprised at how much they have at their disposal.
Start off with Google Analytics (or if they prefer they can use the upcoming free Microsoft analytics tool). You get a wonderful web analytics tool to get going and more. If you do AdWords (Search Engine Marketing) use the new free multivariate testing tool, also free from Google, Web Optimizer. You can use it for A/B testing if you want. In 45 minutes you have gotten ready for the big boys with some pretty nice tools. Invest in people who can use this new found power.
You have a book coming out soon. Can you tell us about that?
The book is Web Analytics: An Hour a Day and it is coming out mid-May 2007.
The spirit of the book is to teach you to fish, rather than giving you a pre-cooked meal. Web Analytics is challenging and every business is unique. My hope with the book is to teach you how to “think web analytics”, to stretch the definition of what web analytics and what it can be. Even the most basic person will find it easy to understand and find insights, at the same time the most advanced users will find specific tips and recommendations that they can apply to elevate the quality of their efforts.
I am also very excited about the book because I am going to donate all the proceeds from the book to two charities, The Smile Train and Doctors Without Borders. So it’s a book that will hopefully help you get most from your data, and you will help me in my efforts to raise money for my charities.
That's quite generous, Avinash. I hope all our readers buy your book -- it would provide great benefit both to their websites and to these two fine charities.
Many thanks for being generous with your time with us today.