Two excellent posts last week about incrementality and matchbacks.

Here’s the gist of matchbacks: if customer gets your marketing message and then orders, the marketing message was responsible for the order. Logical, yes?

No. To establish incrementality, you need a holdout cell to find out the fraction of similar prospects who would ordered anyway. The benefit of the campaign is the sales lift between recipients and control, not total sales to the control.

Here’s Jim Novo talking incrementality at HSN. (Read the whole post to learn the equivalent of SEO for DRTV.)

When the campaigns included coupons, the redemptions were absolutely huge. That’s good, right? Well, in a word, No. Think about it. There was barely any lift in sales at all, yet huge numbers of coupons were redeemed. Meaning?

This means that virtually all the coupons were redeemed by current customers, and the coupon / response did not change their behavior. They bought at the same rate as they would have without the coupon. It means we gave a ton of margin away in addition to the cost of the Campaign, and generated no increase in Sales. We literally would have been better off (financially) by doing absolutely nothing.
–Jim Novo, Marketing Productivity Blog

Here’s Kevin Hillstrom giving a hypothetical from the recent election.

Traditional multichannel marketing was proven as viable via the matchback algorithm. Folks would mail 26 catalogs a year, then take credit for all of the online and retail orders from customers receiving the catalogs.

Mail and holdout tests seldom defend this style of analysis and attribution.

Try this one on, for size. If we believe that matchback analytics are accurate, then the Democratic Party could have sent 60,000,000 postcards to prospective Democratic voters two weeks before the Presidential Election — and then matched each vote back to the postcard. The Democratic party could prove, via matchback analytics, that the postcard was responsible for the election results, right?!
– Kevin Hillstrom, MineThatData

In these challenging economic times, with an abundance of overlapping marketing messages and rampant discounting, campaigns need to show incremental lift, not just sales.

Technorati Tags: control, drilling down, holdout, hsn, incrementality, jim-novo, kevin hillstron, marketing productivity, marketing test, marketing-lift, minethatdata, multichannel