THE RKGBLOG

Attribution Management: 3 “Solutions”

Last week we outlined some of the many problems advertisers face in attempting to untie the Gordian Knot of credit attribution. To summarize: The data is inherently incomplete; integrating data from various sources is messy; and finally, what to do with that data isn’t obvious.

We know that our dashboard view of web sales is wrong. Even online-only pure-plays suffer from cannibalization between channels; those having significant offline marketing activities realize that much of their online business is attributable to those efforts. Web analytics systems should provide attribution management functionality, but often their solutions are not well designed, are too expensive, and don’t provide driving instructions for those in charge of each channel.

The goal of an attribution management system is to at least give us a reasonably accurate and actionable view of how much business is truly driven by each marketing channel so that advertisers may allocate resources productively.

The systems described below will serve this need for better or worse, and should be chosen based on the size of the problem and the cost of the solution.

THE BASIC SOLUTION:

Any system that prevents two marketing programs from taking credit for the same sale is an important start. If each marketing manager or vendor drives only by their own view of sales then double counting is inevitable.

A first touch model is attractive conceptually. Giving credit to the first exposure a buyer had to your brand makes sense. The trouble is defining “first”. If someone first came to your site on an organic search for a “lamp” two years ago, should that touch get credit over the same person’s comparison shopping engine (“CSE”) click on a “sectional sofa” ad which immediately preceded their order? If we place a time limit on credit, say 30 days or whatever, are we then really giving credit to the “first” touch?

A “last touch with brains” credit model may be adequate for some. With brains simply means counting the last paid touch ignoring navigational brand search. For most advertisers last touch on a brand link, whether paid or organic should never be credited with driving a sale. Adding that one caveat to a last-touch system can tremendously clarify the picture.

Careful thought should be given to whether competitive organic search and social media activities should be classified as “paid channels”. The answer may depend on the resources devoted to those efforts, both human and financial.

STRENGTHS:

  • Simplicity. It doesn’t take a ton of coding to pull this off
  • Giving all the credit to the last paid ad is clean, defensible, and
  • It prevents double-counting

WEAKNESSES: Affiliates and emails tend to cannibalize sales from other channels as users look for coupons before they make a purchase. Twitter and Facebook promotions likely have the same effect.

  • Giving all the credit to those coupons ignores the channel that drove the person to the site initially. We’ve seen this cause advertisers to underspend on search and CSE’s which causes not only a drop in sales from those programs but also sales from the cannibalizing channels! Pulling back on the purchase initiating channels leads to fewer sales for the cannibals to eat.
  • This also completely ignores offline media, which for many firms represents the lion’s share of their marketing.
  • In cases where more than one channel played an important role in generating the order only one gets the credit.

MORE SOPHISTICATED APPROACHES

More sophisticated modeling should get us closer to the unknowable truth. Any sophisticated approach should include the following functionality:

  • Flexible Classifications: The ability to apply different rules to navigational brand searches than competitive non-brand searches, both paid and organic, is important. Differentiating between display impressions and display ad click-throughs is crucial. For many, separate classifications for email based on whether an email was received, or the buyer clicked through to the site will add power to the modeling. Having the flexibility to break up channels is essential.
  • Data Imports: The more data fed in, the more accurate the picture. A good model should be able to incorporate data from display ad platforms on view-through impressions which on-site tags can’t capture. Folding in direct mail history gleaned after the order has been placed adds even more valuable insight.
  • Data Exports: The system should be able to feed information out to other systems/vendors at the order level, so that all marketing programs drive by the same view of channel productivity.

COMPLEX HEURISTIC RULES

Parsing credit between paid marketing channels based on a complex set of rules is one way to tackle this in a controlled fashion.

Rules can take many forms, but a few seem obvious:

  • Time: It may make sense to give less credit to ads as they age. Say Fred makes a purchase today. Fred came through a paid search ad an hour before making the order, and two days before that came through a CSE ad. Sally also placed an order an hour after clicking through on a competitive paid search ad, and previously visited the site after clicking on a CSE link, but her CSE visit was 25 days ago, not two. Some time devaluation formula would allow Fred’s and Sally’s orders to be parsed differently.
  • Order: As discussed previously, some marketers may wish to place more value on the first touch than the last, others might see it the other way. Ordering rules allow advertisers to define a function that describes their preference. More credit to first touch AND last touch with less to the middle? Sure. How much more? Complex functions are possible with varying degrees of cursing from the engineering teams.
  • Special Cases: For those channels likely to cannibalize orders, special rules can be concocted to ignore or devalue touches, for example if they follow another paid touch within X minutes.

STRENGTHS:

  • Flexibility and control: The advertiser can value conversion paths as s/he sees fit.
  • Comprehension: It’s easy to understand why credit for a given order has been split the way it has, and if scrutiny reveals unexpected and unwanted effects, the model can be adjusted easily.
  • Fractional credit makes sense, and should provide a better picture than simple first touch or last touch systems.

WEAKNESSES:

  • It relies on gut instincts. On what basis are the rules defined? Marketing intuition is a pretty good starting place, but many folks would look at the above controls much like the dashboard of a submarine and with about as much confidence in what buttons they should push.
  • Absent sufficient confidence, marketers may hesitate to drive by those numbers greatly reducing the value of the system.

SOPHISTICATED STATISTICAL MODELS

Building a mathematical model to “solve” the problem can be done. Using Hidden Markov Chains and other, more flexible, dynamic Bayesian networks, we can build a unique model for each advertiser that assesses the influence of each ad on an order.

STRENGTHS:

  • The modeling eliminates the need for intuitive judgment required by a heuristic model.
  • Confidence in the methodology leads to action.

WEAKNESSES:

  • Fancy models can provide a false sense of precision. Statistics is as much art as science, and in the wrong hands powerful techniques can produce lousy results.
  • Because of the complexity, it is often hard to spot problems with a model’s design. Answering the question: “why was this order attributed in this manner” can result in the frustrating answer “because the model says so…don’t question the model!”
  • The return on the extra investment may not be worth it. Building custom models takes time, know-how, and often a good bit of computing horsepower. If the results aren’t very different from a much more straight-forward model it could be wasted effort.

THREE CRITICAL CONSIDERATIONS:

  1. The best, most reliable and accurate information on the ROI of a marketing program comes not from attribution modeling but from hold-out tests. By running carefully designed tests we can very accurately determine the lift produced in online orders by a catalog mailing, an email blast, and display advertising. Indeed, feeding the results of those tests into the more complex models and using them as “tuning forks” is imperative.
  2. The implications of #1 include the notion that at the end of the day the principal role of the attribution system is really to determine the incremental value of competitive paid search, CSEs, affiliates, social media efforts and organic search efforts, and perhaps to reduce the frequency of ongoing hold-out tests for the testable channels.
  3. Costs and benefits of the attribution management system need to be carefully balanced. If no attribution system exists presently, the value of going to a “last touch with brains” model will be large. For advertisers who already have such a model the principal value may turn out to be identifying affiliate cannibalization and fine-tuning credit and/or efficiency targets for paid search programs. If those programs are large and the managers of those programs have the ability to react smartly to the results the value of a more powerful attribution system could be substantial. The cost of the system shouldn’t outweigh these benefits.

In our next post, we’ll outline the RKG attribution management system and discuss ideas for pricing the service appropriately.

Comments
12 Responses to “Attribution Management: 3 “Solutions””
  1. André says:

    Thanks Goerge, I’m really looking forward to read the next post!

    Ever thought about multiple buying decision processes running at the same time, relevant for the same retailer (e.g. one who likes to buy a camera and a shirt)? I’ve written my master-thesis about 2 1/2 years ago about credit attribution and encountered the semantics of the keywords/ads (or touch points to be more general) to be highly relevant; using an ontology allows to better determine the proximity of the touch points and the conversion (especially for retailers).

    But as you mentioned, costs can rapidly outweigh benefits (which at the end made us to only implement parts of the things we found)…

  2. Terrific question, Andre,

    We have thought about it, and studied it some. A post I wrote a while back on the paid search buying cycle showed that in many of the cases we examined, the first ad was completely unrelated to the last ad, and in each of the cases we studied the order was related to the last ad, not the first.

    This leads to one of two conclusions: 1) the first visit was worthless as the person clearly bought what they searched for from another vendor; or 2) the first visit exposed someone to my brand, hence when that person saw my link after the other search they selected it preferentially based on that past positive, if not lucrative experience.

    Between the complexity of that argument and the difficulty of dynamically determining semantic relationships between a user search and the items on the order — “Dog chews” searcher buys “Greenies” — we’re letting that one lie unless someone wants to pay us to dive in. I just can’t imagine the improvements would be worth the cost.

  3. Julian says:

    A very nice and thorough post, George!

    You highlight quite well the complexity we’re dealing with and I’m looking forward to seeing any ideas for a simple reporting solution to this. Maybe there is none and we need a whole bunch of reports to understand and attribute activity.

    As a side note: when I looked at this I was (hopefully not for long) missing two bits of data: new vs existing customer journeys and non-converting data.

    I think new and existing customers may have very different routes to conversion. Putting them together may produce a worthless average?

    As for non-converting data – I think that’d be great for the modelling you discuss towards the end. Surely we can correlate marketing activity with the final outcome (sale or abandon) and that can help us with decisions.

    Either way, very much looking forward to your next post.

  4. André says:

    You might consider to use the retailer’s internal recommender system / onsite search engine to check whether a item one has bought is immediately related to one or more search phrase/s (or ads) the user has used (contact to) before. This is the approach we followed. But nevertheless, it is very quickly becoming very complex…

  5. Andre, our tracking actually provides us with the item names on each order, so the data isn’t hard to collect it’s just hard to parse systematically. For a case by case study “eyeballing it” makes sense, but building a dynamic process to do it accurately on the fly is tough as you’ve discovered!

    Julian, thank you for your kind words. The new buyer vs returning customer issue is a great one, but the actionable information derived isn’t clear to me. Knowing that a preponderance of new buyers come from some channels not others is useful for adjusting efficiency targets — advertisers are likely to spend more to attract new customers than to generate repeat purchases — but since we can’t really target ads to users based on whether they’ve bought from us previously or not (the ad serving platforms can’t know) I’m not sure how much value there is in seeing the different paths. And, it seems like most folks already have a sense that affiliates drive few new buyers relative to competitive paid search.

    The second question relating to conversion rates is a tough one. We have that information for any ad interaction that involves a visit to the website, but in general won’t have it for display ad impressions or emails sent without the recipient clicking through to the site. We could in theory accept those null visits in with the other data but we’re not sure it adds value, as both those channels are testable without the modeling.

  6. Julian says:

    Well, knowing what works for new versus existing customers can be actionable information. Let’s say that a company suffers from declining retention rates – where should it invest to reverse the trend? Email and affiliates are obvious channels, but what about Remarketing (e.g. Criteo), does that help? Alternatively, if I want to expand and acquire new customers shall I spend my money on PPC, Display or is Remarketing more effective? (Of course, this is all on average, there are few channels that you can target directly to new customers, as you say.)

    Anyway, I’m going slightly off-topic here. My point is that looking at attribution reports while ignoring the user group they’re describing may produce inaccurate results. I suspect some extremely active customers that interact through all digital channels can mask the few tentative interaction by a dozen new customers. Therefore attribution reporting AND customer segmentaion sounds like a good idea, but now that will definitely crash my Excel.

    Sorry to muddy the already muddy waters. As I said, looking forward to the next post!

  7. I’m with you, it’s definitely important to know which channel drives new buyers vs repeats. I’ll be surprised if the impression folks have now of that breakdown is impacted very much by attribution modeling, but as you suggest, we should do the analysis to make sure.

  8. Zach says:

    I’m thinking attribution is messy and not about to get any clearer any time soon. Why not a more holistic view where you add up all costs and all conversions to come up with an overall ROI and Cost/Conv. Then test spending in different ways (let’s try adding 20% to display this month) to see how it effects the overall result.

  9. For the expensive/testable channels like display and direct mail, hold-out tests are the way to go. Using public service announcement controls for 10% of the impressions will tell you how much genuine lift each display strategy or each direct mail segment creates.

    Sequential pushes and pulls as you describe tend not to be terribly revealing. Delay between interactions with ads and orders can cloud the picture, but also just general swings in the business unrelated to marketing as well.

Trackbacks
Check out what others are saying...
  1. [...] This post was mentioned on Twitter by Alan Rimm-Kaufman, Jeremy J Brown and Search Marketing, PPC Marketing. PPC Marketing said: Attribution Management: 3 “Solutions”: Last week we outlined some of the many problems advertisers face in attempt… http://bit.ly/9Epf3m [...]

  2. [...] the last couple of weeks we’ve detailed both the challenges of attribution management and some of the approaches used to address these problems. Today, we outline our [...]