Brand Ad Cannibalization: A Tale of Two Tests
My monthly paid search column at SEL, in case you missed it:
Should companies pay for ads on their brand name?
There are plenty of good reasons to do so regardless of the ROI, but those keenly interested in ROI and vexed by the concept of paying for traffic on their trademarks still ask the question.
We’ve done some research on this topic that we’ll share below. We focused on trying to answer a couple of specific questions:
- Do brand ads cannibalize traffic from organic listings? and
- If the answer to the first question is: “it depends”, on what does it seem to depend?
First let’s define our terms. What do we mean by ads cannibalizing traffic?
The impact on traffic and eventual revenue of combining ads with organic listings can take many forms.
A concept that might be useful to discuss is: The incremental traffic ratio (“ITR”) of ads. That is the fraction of the reported clicks on an ad that are in fact incremental.
Let’s say that absent an ad, we expect 1,000 clicks on an organic link on a given day. Ads could have 3 general effects on traffic.
- Destructive Cannibalization: we flip on the ad and find that the overall traffic declines. For example:
The ITR is the ratio of incremental traffic to measured ad traffic, which in this case is -200/400 = -50%
This might also be referred to as “1 + 1 = 0.5″
- Constructive Cannibalization: presence of the ad increases the overall traffic volume, but some of the traffic on the ad is ‘stolen’ from the organic listing, so the ad can’t take credit for all of the traffic it sees, only the incremental traffic driven. For example:
Pretty clearly, Constructive Cannibalization can range from one edge case we’ll call No Cannibalization whereby 100% of the traffic on the ad is incremental, to the other edge case of Perfect Cannibalization where the presence of the ad doesn’t destroy traffic, but doesn’t increase it either. In the language of ITR, Constructive Cannibalization exists when the ITR is between 0% and 100%.
Clearly, somewhere in the Constructive Cannibalization ITR range from 0% to 100% the advertising spend on brand ads likely shifts from a negative ROI to a positive ROI.
- Amplification: The traffic through the sponsored link is 100% incremental AND the ad is responsible for increasing the traffic on the organic link as well. For example:
The best of all scenarios, the ad creates more love all around.
When ITR is greater than 100%, an ad is having more positive impact than what is readily apparent.
Many folks with an agenda conflate Constructive Cannibalization with Amplification, using the fact of incremental value greater than 0 to imply that 1 + 1 = 3. Amplification may exist in some instances, but the existence of incremental overall traffic doesn’t imply amplification.
Designing a Good Test
First, we must define what constitutes a brand ad and think about whether there are different classes of brand searches. There may be some search phrases that clearly demonstrate navigational intent: they’re looking for you! There may be others where the navigational intent is less clear: some are looking for you, some are looking for something else with a similar name. Classify these ads by type so that we may see whether the incremental value of the ads is perhaps higher when the navigational intent is less clear. Also, classify the corresponding organic listings by position on the page. Hopefully, your organic brand links are at the top of the page, but it isn’t always the case if your brand is also a competitive search term for others (eg “online poker” is both a brand and a search term for the category).
Second, establish the measurement mechanics. You need to be able to see traffic by keyword by source (which engine). It’s best to use the same platform (attribution system or analytics package) for consistency.
Third, establish the measurement metrics. It’s best to measure conversions within X hours of a click as a benchmark. However, conversions are a much thinner data set than click traffic. Reaching statistically significant data can take much more time if one insists on using conversions rather than traffic, so traffic with an understanding of average conversion rates may get the most valid results withing the shortest period of time.
Fourth, establish the measurement methodology. Unless you actually serve the ads (that is, unless you’re Google or Bing) the only way to do this is by turning off ads and watching performance carefully. It’s best to cycle ads on and off several times to try to separate anomalous performance (I feel compelled to use the term “exogenous factors”) from cause and effect. One hour on, one hour off, one hour on, one hour off; or week on, week off, week on, week off, etc.
Fifth, make sure off is “off”. It’s best to not just pause the brand ads but to add brand phrases as account negatives to be sure that no ad is showing.
Sixth, decide how you’ll analyze the data. How will you account for expected hourly, daily, weekly, seasonal variations in search volume. A couple of good ideas: benchmark against Bing. In other words, only conduct the test on Google, and use the Bing brand search (paid and natural) as a control. Overall site traffic can be a control as well. This is not trivial, but with sufficient data you should be able to get a very good sense of the interaction without having to hire a statistician to perform mathematical wizardry. Deciding how you will look at the data before you collect it helps keep you honest, and prevents the analyst from cooking the books to produce the desired outcome, whatever that might be.
Seventh, run the test.
Our Test Results
We performed two such tests for different clients recently, and the results were very interesting, and not what we expected.
We expected to find that brand ads cannibalized all or almost all of their traffic from brand organic listings. We didn’t expect Destructive Cannibalization but we did think we’d see ITRs of less than 10%.
In the first test, we were STUNNED to find that overall 67% of traffic on brand ads was incremental. Only about a third of the traffic appeared to be cannibalized from the organic link. Interestingly, the phrases we categorized as clearly navigational did have a markedly lower ITR (55%) than those that were more ambiguous (ITR = 90%). However, 55% ITR for navigational searches was still much, much higher than we expected to see.
The second test turned out QUITE differently. The data was somewhat thinner, and we'd like to have more to work with for this one, but from every angle the ITR of brand ads was essentially 0%. In this case the advertiser had never advertised on its brand, and we ran ads for a period only to find the traffic to be pretty much completely cannibalized from the organic listing.
The hypothesis we want to test is this: The more navigational the search the less likely the traffic is to be incremental. However, if outside factors make it difficult to navigate to your site, brand ads can absolutely bring in incremental traffic. In the second test, the advertisers' brand name was very unique and unlikely to be confused for anything else. There are no other ads on the page, there are no close cousin websites that might create confusion, hence the ad provides navigational help that no one really needs.
In the first test, the advertiser’s name is closely connected with a VERY well known, related entity. There are many other ads on the page for brand search and many cousin links that could draw in users easily. Here the navigational assistance provided by an ad is much more necessary to make sure you get the traffic that is trying to find you.
We’d expect that advertisers with wide distribution networks like product manufacturers, entertainment, and travel will find more incremental value in brand advertisements in that there will be many other companies competing well for that traffic. “United flights to Las Vegas” would prompt ads from many OTAs with attractive offerings, not just United Airline’s ad.
Since N=2, here, we really can’t say we’ve done much more than establish an hypothesis for further testing. We do believe we’ve defined a repeatable methodology for studying these data in a sophisticated and useful fashion going forward and that this methodology can be extended to non-brand competitive search as well. If 10% of sales attributed to a paid search ad are cannibalized from an organic link, doesn’t that change its ROI? If, in non-brand search, ads and organic links amplify one another, isn’t the ROI better than what it appears to be? Shouldn’t we act on that information?
One conclusion we can draw definitively is: answers lie in the data, not in marketing blather. Average values are meaningless as each case varies. If you wonder about whether you should or shouldn’t advertise on your brand there is only one way to find out.
The statistics to tease out exogenous effects ‘properly’ is non-trivial, but don’t let the perfect be the enemy of the good enough. You can get pretty close to the ‘right’ answer with smart application of basic math.