Last week we outlined some of the many problems advertisers face in attempting to untie the Gordian Knot of credit attribution. To summarize: The data is inherently incomplete; integrating data from various sources is messy; and finally, what to do with that data isn't obvious.
We know that our dashboard view of web sales is wrong. Even online-only pure-plays suffer from cannibalization between channels; those having significant offline marketing activities realize that much of their online business is attributable to those efforts. Web analytics systems should provide attribution management functionality, but often their solutions are not well designed, are too expensive, and don't provide driving instructions for those in charge of each channel.
The goal of an attribution management system is to at least give us a reasonably accurate and actionable view of how much business is truly driven by each marketing channel so that advertisers may allocate resources productively.
The systems described below will serve this need for better or worse, and should be chosen based on the size of the problem and the cost of the solution.
THE BASIC SOLUTION:
Any system that prevents two marketing programs from taking credit for the same sale is an important start. If each marketing manager or vendor drives only by their own view of sales then double counting is inevitable.
A first touch model is attractive conceptually. Giving credit to the first exposure a buyer had to your brand makes sense. The trouble is defining "first". If someone first came to your site on an organic search for a "lamp" two years ago, should that touch get credit over the same person's comparison shopping engine ("CSE") click on a "sectional sofa" ad which immediately preceded their order? If we place a time limit on credit, say 30 days or whatever, are we then really giving credit to the "first" touch?
A "last touch with brains" credit model may be adequate for some. With brains simply means counting the last paid touch ignoring navigational brand search. For most advertisers last touch on a brand link, whether paid or organic should never be credited with driving a sale. Adding that one caveat to a last-touch system can tremendously clarify the picture.
Careful thought should be given to whether competitive organic search and social media activities should be classified as "paid channels". The answer may depend on the resources devoted to those efforts, both human and financial.
- Simplicity. It doesn't take a ton of coding to pull this off
- Giving all the credit to the last paid ad is clean, defensible, and
- It prevents double-counting
WEAKNESSES: Affiliates and emails tend to cannibalize sales from other channels as users look for coupons before they make a purchase. Twitter and Facebook promotions likely have the same effect.
- Giving all the credit to those coupons ignores the channel that drove the person to the site initially. We've seen this cause advertisers to underspend on search and CSE's which causes not only a drop in sales from those programs but also sales from the cannibalizing channels! Pulling back on the purchase initiating channels leads to fewer sales for the cannibals to eat.
- This also completely ignores offline media, which for many firms represents the lion's share of their marketing.
- In cases where more than one channel played an important role in generating the order only one gets the credit.
MORE SOPHISTICATED APPROACHES
More sophisticated modeling should get us closer to the unknowable truth. Any sophisticated approach should include the following functionality:
- Flexible Classifications: The ability to apply different rules to navigational brand searches than competitive non-brand searches, both paid and organic, is important. Differentiating between display impressions and display ad click-throughs is crucial. For many, separate classifications for email based on whether an email was received, or the buyer clicked through to the site will add power to the modeling. Having the flexibility to break up channels is essential.
- Data Imports: The more data fed in, the more accurate the picture. A good model should be able to incorporate data from display ad platforms on view-through impressions which on-site tags can't capture. Folding in direct mail history gleaned after the order has been placed adds even more valuable insight.
- Data Exports: The system should be able to feed information out to other systems/vendors at the order level, so that all marketing programs drive by the same view of channel productivity.
COMPLEX HEURISTIC RULES
Parsing credit between paid marketing channels based on a complex set of rules is one way to tackle this in a controlled fashion.
Rules can take many forms, but a few seem obvious:
- Time: It may make sense to give less credit to ads as they age. Say Fred makes a purchase today. Fred came through a paid search ad an hour before making the order, and two days before that came through a CSE ad. Sally also placed an order an hour after clicking through on a competitive paid search ad, and previously visited the site after clicking on a CSE link, but her CSE visit was 25 days ago, not two. Some time devaluation formula would allow Fred's and Sally's orders to be parsed differently.
- Order: As discussed previously, some marketers may wish to place more value on the first touch than the last, others might see it the other way. Ordering rules allow advertisers to define a function that describes their preference. More credit to first touch AND last touch with less to the middle? Sure. How much more? Complex functions are possible with varying degrees of cursing from the engineering teams.
- Special Cases: For those channels likely to cannibalize orders, special rules can be concocted to ignore or devalue touches, for example if they follow another paid touch within X minutes.
- Flexibility and control: The advertiser can value conversion paths as s/he sees fit.
- Comprehension: It's easy to understand why credit for a given order has been split the way it has, and if scrutiny reveals unexpected and unwanted effects, the model can be adjusted easily.
- Fractional credit makes sense, and should provide a better picture than simple first touch or last touch systems.
- It relies on gut instincts. On what basis are the rules defined? Marketing intuition is a pretty good starting place, but many folks would look at the above controls much like the dashboard of a submarine and with about as much confidence in what buttons they should push.
- Absent sufficient confidence, marketers may hesitate to drive by those numbers greatly reducing the value of the system.
SOPHISTICATED STATISTICAL MODELS
Building a mathematical model to "solve" the problem can be done. Using Hidden Markov Chains and other, more flexible, dynamic Bayesian networks, we can build a unique model for each advertiser that assesses the influence of each ad on an order.
- The modeling eliminates the need for intuitive judgment required by a heuristic model.
- Confidence in the methodology leads to action.
- Fancy models can provide a false sense of precision. Statistics is as much art as science, and in the wrong hands powerful techniques can produce lousy results.
- Because of the complexity, it is often hard to spot problems with a model's design. Answering the question: "why was this order attributed in this manner" can result in the frustrating answer "because the model says so...don't question the model!"
- The return on the extra investment may not be worth it. Building custom models takes time, know-how, and often a good bit of computing horsepower. If the results aren't very different from a much more straight-forward model it could be wasted effort.
THREE CRITICAL CONSIDERATIONS:
- The best, most reliable and accurate information on the ROI of a marketing program comes not from attribution modeling but from hold-out tests. By running carefully designed tests we can very accurately determine the lift produced in online orders by a catalog mailing, an email blast, and display advertising. Indeed, feeding the results of those tests into the more complex models and using them as "tuning forks" is imperative.
- The implications of #1 include the notion that at the end of the day the principal role of the attribution system is really to determine the incremental value of competitive paid search, CSEs, affiliates, social media efforts and organic search efforts, and perhaps to reduce the frequency of ongoing hold-out tests for the testable channels.
- Costs and benefits of the attribution management system need to be carefully balanced. If no attribution system exists presently, the value of going to a "last touch with brains" model will be large. For advertisers who already have such a model the principal value may turn out to be identifying affiliate cannibalization and fine-tuning credit and/or efficiency targets for paid search programs. If those programs are large and the managers of those programs have the ability to react smartly to the results the value of a more powerful attribution system could be substantial. The cost of the system shouldn't outweigh these benefits.