THE RKGBLOG

All Preceding Touches Do Not Assist

My monthly paid search column at SEL in case you missed it:

In the complex world of media mix analysis it’s important to have a trustworthy guide. Just as a last touch attribution model can lead to mis-allocation of resources, over-crediting first touches can mislead as well.

All marketing/advertising impressions are not equally valuable. A thirty-second TV spot, a quality visit to your website, a walk through your brick and mortar business are significantly more valuable ‘impressions’ than exposure to a print ad, a display ad or a text ad. A link (paid or organic) on a SERP for a competitive non-brand search is much more likely drive incremental business than traffic from someone searching for “YourTradeMark Coupons” and coming through an affiliate. These touches shouldn’t all be treated the same, and good attribution systems need to ‘understand’ and/or ‘sniff out’ those distinctions.

Let’s take a look at an example from basketball:

The Bulls led the series 3-2. Game 6 came down to the wire.

Before the action in the video started, Pippen in-bounded the ball to Kerr.

So the path to conversion looked like this:

Pippen => Kerr => Pippen => Jordan => Kerr => Conversion

Let’s take a look at how 5 different approaches to attribution would handle that conversion:

  1. No Attribution System: The silo view would spread credit for the conversion as follows:
    • Kerr: 100%
    • Jordan: 100%
    • Pippen: 100%

    Grade: F

  2. Last Touch Attribution:
    • Kerr: 100%
    • Jordan: 0%
    • Pippen: 0%

    Grade: B-

  3. First Touch Attribution:
    • Kerr: 0%
    • Jordan: 0%
    • Pippen: 100%

    Grade: D-

  4. Proportional Attribution: Crediting each touch equally we’d split the credit this way
    • Kerr: 40%
    • Jordan: 20%
    • Pippen: 40%

    Grade: D+

  5. Assist Tracking Attribution: Crediting the order to the last touch and assists to each preceding touch we’d view this transaction as follows:
    • Kerr: 100% + 1 Assist
    • Jordan: 1 Assist
    • Pippen: 2 Assists

    Grade: C+

Pretty clearly, none of these attribution models provides a very good understanding of that conversion, but any of the attribution models is preferable to none.

‘Assist’ counting can be particularly misleading over time. Consider our basketball metaphor, extended. Pippen always in-bounded the ball. Blind assist counting will lead one to conclude that he was the greatest play-maker in history, averaging 50, 60, maybe 70 assists a game since he often touched the ball on the offensive end of the court as well!

Consider another case. Suppose someone develops the ultimate blanket advertisement. An ad for Acme pops up on every computer, every mobile device, every TV screen in the country on boot-up. Was every conversion on Acme’s site that day impacted by those ads? Would the site have had no conversions absent the ads? Of course not.

What we’re really interested in learning is not what ads consumers were exposed to, but what lift can be credited to those ads.

A better metaphor might be the plus/minus ratio in hockey. When this player was on the ice, did our team perform better or worse and by how much?

Pure A/B split tests for email, display ads, and direct mail provide the cleanest answers to those critical questions. Unfortunately, pure testing isn’t possible in paid search, natural search and other less trackable forms of offline marketing. It is possible to hack at the incremental value of paid search through testing, but those tests can be challenging to design and execute, and incur material opportunity costs.

Attribution systems help defray the cost of ongoing testing. We believe firmly that periodic A/B testing remains crucial to calibrate coefficients for attribution systems particularly for display advertising, but attribution allows advertisers to reduce the need for ongoing tests significantly.

As we pointed out last year, mathematicians without guidance from marketing experience will build the wrong types of models. Bayesian models tend to over-credit affiliates, email and brand ads because visits to these just before a purchase strongly correlate with conversion success. As marketers, we recognize that the cause of this correlation relates to the unique manner in which consumers use coupon sites, email offers, and navigational search.

Building a smarter statistical model is a heck of a challenge. We want a model that more closely matches our intuition as marketers without unfairly biasing the outcome towards one channel or another. We want a model that recognizes cannibalistic patterns, recognizes the difference between display impressions and display click-throughs,* and handles behaviors associated with some channels more than others. As an example of the latter, we see consumers sometimes bang through 5 or 10 affiliate ads in the space of a few minutes looking for the best offer, and would argue that channels demonstrating that type of sequence shouldn’t get 5 or 10 bites out of the apple. This is not simple, but can improve our perceptions of how well our marketing efforts work for us and lead to better resource allocation.

*Note: This is not to suggest that display impressions are meaningless. Far from it. Well-targeted, thoughtful display advertising with in-market data and smart Real Time Bidding can be a terrific means of driving business cost effectively. It is also imperative to understand that the lift in traffic from display advertising is mostly produced by impressions, not direct click-throughs. Those impressions really do have value. That value just needs to be measured carefully.

At the end of the day attribution’s real value lies in providing better insights to drive by. Managing paid search based on attributed conversions rather than silo-view matters. Paying commissions to CSE vendors and affiliates based on attributed conversions can save a ton on unearned commission.

Teasing insights out of conversion path data can prove useful as well. One client of ours believed that 40% of orders ‘driven’ by affiliates were from new-to-file customers. Studying the data more carefully revealed that most of the new-to-file customers credited to affiliates had come to the site previously through search. Affiliate-only conversions were existing customers 92% of the time!

The interrelationships between marketing activities creates new and exciting challenges for marketers from every channel. The more we understand and embrace those challenges, the more we work together instead of in silos, the more successful our businesses will be going forward.

Comments
10 Responses to “All Preceding Touches Do Not Assist”
  1. Ethan says:

    Great post, as usual! I wrote a longer comment, but it got lost. I’d give first touch better than a D, re NTF and incrementality. I also like multiple lenses. The gap by channel when looking at 1st, last and any touch is insightful. Along with goals and points, there is also a story (potentially non-linear). Have you seen this new camera (link below)? You can click a photo (in a browser) and dynamically focus on different parts of the picture. Same photo/story, with different 2D lenses. The example photos are at the bottom of this link… click on the pics to refocus the image.

    http://www.petapixel.com/2011/06/22/lytro-is-developing-a-camera-that-may-change-photography-as-we-know-it/

  2. Thanks Ethan,

    Sorry about the squirrely comment field! It is interesting to look through multiple lenses — and I’d like to invest in that camera company! Wow is that cool! There’s another camera out there that is similar but the notion is taking the same picture with all different aperture/lense speed settings at the same time allowing you to adjust after the fact. http://www.rimmkaufman.com/rkgblog/2009/10/13/ppc-click-volume/

    I gave first touch a D in this case because it was Pippen in-bounding the ball — no bearing on this play. Metaphorically, like someone visiting your site through your organic brand link, then a month later clicking on a competitive paid search ad for “cotton hoodie”.

    Thanks for your comment!

  3. Evan says:

    Great post George!

  4. Thanks Evan, and thanks for stopping by our humble blog!

  5. Tim Peter says:

    Hi George,

    Great post, but I have a question. You say, “Bayesian models tend to over-credit affiliates, email and brand ads because visits to these just before a purchase strongly correlate with conversion success.” Shouldn’t you be able to adjust the priors for these sources in your model to account for the undue weighting they tend to receive? I’m looking at some Bayesian models right now and think this is likely the best approach based on the availability of data from the various sources. Or am I missing something obvious?

    Other than that, keep up the great work.

  6. Hi Tim,

    Yes Bayesian approaches can be used with adjustments for specific channels and sub-channels. The issue is: if we’re hard wiring in suppression coefficients for certain channels are we really letting the data speak for itself, or do we end up with an essentially heuristic approach? We’re using statistics to get away from guess work, so if we use guess work under the hood…There isn’t a perfect solution. We like our approach, and the credit attributed order by order generally makes sense. I have no doubt that there are many ways to skin this cat, but there are also many considerations that need to be addressed.

    Thanks for the excellent comment and let us know how you fare!

  7. Dylan says:

    I generally like first click attribution model. The challenge with using last click attribution to our business is the life cycle of the product. Our Adwords traffic get converted into registered users (one of the conversion goal) and they don’t complete their product customization until many weeks or even months down the road. And since our typical user is a 40 years old woman, lots of the users typed in our trademark in Google and click on our PPC ad or Organic link to reach our site and finish their product days or weeks down the road. And they most likely visit the site that way several times to complete the product customization in different days. Since Adwords uses last click attribution model, the revenue is attributed to the trademark search query instead of the original non brand query that generated the registered user.

    How would you approach attribution in this case?

  8. Dylan, thanks for your comment. As we always say: your mileage may vary, so you have to look at your own data. Generalizing is dangerous. We’ve rarely found the effect you describe: non-brand ad followed by a brand ad to account for more than 5% of last-touch brand conversions. Our platform by default gives credit to the last non-brand ad in these cases and that’s probably the best way to handle it: give preference to competitive search any time it appears in addition to a brand ad touch. Fractional attribution is the gold standard, but ignoring trailing navigational touches (brand paid or organic clicks) is a pretty good hack.

Trackbacks
Check out what others are saying...
  1. [...] Michie and the RKG team. Their RKBLOG is thoughtful, insightful and colorful. Need proof? Check out their recent piece on attribution modeling and see if you don’t want them on your team, [...]

  2. [...] to get used to how to make the most of these reports. As George Michie pointed out a while back, not every touchpoint in the marketing funnel assists conversions. Still, this is an important step forward for Google Analytics and one that every site owner should [...]