Dec 222008

More isn't always Better

Occasionally, RKG is taken to task by the engines for not towing "the Party Line" with our clients.

The Party Line is essentially:

"Don't get too caught up in measuring ROI in your paid search efforts. There are so many factors that make it impossible to measure accurately, like phone spillover, like channel cannibalization, like retail spillover, like cookie breakage, that you'll grossly underestimate the real value of search. The value to brand building of being in the top 3 spots on high traffic keywords is huge and while the benefits may be hard to trace, they're large. Retailers should just spend more."

To bolster these claims, the engines and some agencies who tow the line with them often cite studies suggesting the massive connection between online research and offline purchasing behavior. The In-Store Marketing Institute has published a study along these lines, and someone has used slides from Forrester on this subject at every conference I've been to in the last year or so -- usually the same slides -- citing something on the order of 6 offline orders driven by online research for every online order placed.

But what does this shopping behavior mean for marketers? Obviously, for pure plays and catalog firms with few physical stores, not much, but even for brick and mortar chains: how should you adjust your advertising behavior given these data? Our advice is: very carefully.

The problem with the research is it really doesn't answer the questions that matter regarding the influence of online ads, namely:

  • What fraction of the offline purchases went to the same company on whose website the research was done? Some fraction of these people likely already know where they're going to buy the product, they're just trying to figure out which model to buy, and any website that has good product comparison functionality will do. It doesn't warm the cockles of my heart to have my advertising dollars and web design helpfully guide people's purchases from my competitor's retail store.
  • Of the folks who do buy at one of your retail stores after visiting your website: how did they find the website? Did they simply load the site directly, or search for your site by name? If so, your advertising dollars aren't responsible for those offline orders, the power of your brand drove those orders.
  • How many of the folks doing "online research" just used the website to locate the nearest store, or print off a coupon?
  • Once you've discounted the extra orders driven by the above, what fraction of the remainder found your site through organic links rather than your paid advertising? No sense crediting advertisements if the "free" listings deserve the credit.

At last summer's Internet Retailer Conference I was impudent enough to get up and ask the speaker from Forrester some of these questions, and was quickly silenced with harrumphs to the effect that the data was: "directionally accurate." As a physics student I learned that vectors have two attributes: direction and magnitude. If you know the direction, but not the magnitude, you don't know much.

We all know that online advertising influences some offline purchasing that is helpful to the advertiser, just as offline advertising drives tremendous amounts of online activity. What we don't know is how much.

There is no question that the puzzle is complicated, with the publishers in each channel claiming credit for the orders driven in the other channels.

We, at RKG, know that the publishers' interest and the advertiser's interests are different, and we line up on the side of our clients. We understand that the total marketing spend in all channels can't exceed a certain fraction of top line revenue in all channels, and that the key is to measure what you can, and test different mixtures carefully to achieve maximum ROI.

As geo-targeting becomes more robust, testing a big push in limited geographies around retail hubs may help gauge the magnitude of that spillover. Testing a collection of geographies with a few weeks "on" a few weeks off, with half toggling opposite to help reduce seasonal effects might give a sense of what that spillover looks like. We know it's there, we believe it's material, but ultimately we believe in testing the water before diving in head first.

We'd love to hear your thoughts on the subject!



6 Responses to "More isn't always Better"
Rick Galan says:
Even doing specific geo-targeted tests really don't get us all of the way there though. There is so much fluctuation in consumer behavior based on product line, brand, specific offer, etc. that I think the best we can accomplish with current tracking is getting that direction attribute better hammered down. Before we will be able to really understand the magnitude, I think one of two things has to happen: 1: A Nielsen-like system has to be put in place that not only monitors online behavior but offline spending and behavior as well, and correlates the two. A large enough sample size for that group would likely get us in the ballpark of understanding the magnitude. 2: Online and consumer tracking develops to the point where an online and offline identity can be established using cookies and credit cards, or perhaps individual stores can use the phone number (like RadioShack who always asks). I have no doubt that this type of "Minority Report" consumer tracking is coming, but obviously still down the road a bit. :) In the meantime, I'll just know that there is an added branding bonus that comes from search marketing, and not let the people who stand to gain the most tell me where to put my money. :) -Rick
Jim Novo says:
George, I have to ask the same question I asked Alan on his "melons" post - how do we get people to care about profits? Every time I say that it makes me shiver, but the truth is nobody will be interested in these kinds of measurement challenges until the goal is profit, not sales. As long as the goal is sales, more = better. This issue of the "true incremental" of a marketing program - how big is natural or organic demand when you take away the ads? - is hiding right around the corner, and when (if) it is embraced, there will be a huge body count in Online Marketing. The BI folks know all about this issue, and it will be one of the first disciplines they bring to web analytics, I think. How many people use control groups in their e-mail programs? What is the true lift of these programs, and how many people "would have bought anyway"? When you do the real math, including the value of discounts, what is the subsidy cost of these e-mail programs among best customers? At HSN, we went through years of testing the effects of offline media on performance. These were media market tests of the type you refer to above, except that we always had a control media market to account for variations in the environment. Billboards. TV. Radio. Newspapers. TV Guides - local, national, and cable. Flyers, Shoppers. Spot cable. All of it, in just about every combination you can think of. We could never pay the media back on a net-bet basis; it generated some demand, but not enough to pay out. The fact we were already a TV media, combined with the interactivity of the program itself and TV search engine (the remote control) was more powerful than any media we could buy. The single most effective "offline" tactic was to move our channel position closer to the major networks on the cable system, so that when people flipped channels, they stumbled over our channel more often. This move increased Natural demand, which came from the interactivity of the TV show when people flipped "through" it. In effect, this was SEO for TV. All about "ranking", being where the eyeballs are. The same natural demand issue is happening on the web, but nobody is measuring it. Because profits are not the goal. Yet?
Not all of our clients do think this way. Many do, but others are more concerned about share of voice, and the trappings of a good program. We still speak our language because many find it useful, and we add value where we can. Often times we find that our day to day contacts get it, but the corner office doesn't. I still think the ultimate measure is incrementality, but you're right, not everyone wants to here that message.
Vijay R says:
With mobile phones being ubiquitous, why not use them as a channel to measure effect of online on offline sales? For instance, offer a special discount, or upgrade, to be sent as a coupon via text message to the customer's phone. At the time of purchase, the customer has to show the message, and this can be fed into the existing tracking systems. This is far less cumbersome than having to print coupons. I'm surprised that retailers don't make better use of customer's phones. While the nirvana of always-on geo presence has been promised for ages, there are other methods available which should leverage the fact that the phone is always with the customer.
Vijay, that's the best idea I've heard in months! Rick, your point is well-taken. It's really noisy, and for many retail chains online media buys are such a small portion of their overall advertising efforts that it may be impossible to measure the impact even with a great tracking mechanism like Vijay's. However, if you can't separate the signal from the noise, maybe that speaks to the degree of impact on consumer behavior relative to the other stuff? Thanks for your comments!


Check out what others are saying...
[...] off a great post by George on marketing measurement, here’s a very specific example of how Marketers have to think [...]

Leave A Comment