We presented some original research on paid search click streams at last month’s
SES show in New York.

Here’s what we learned.

Like many folks, when we manage and optimize paid search campaigns, we do so by computing the effectiveness of each ad. This means comparing the cost of each ad to its corresponding tracked sales.

There’s more complexity involved than that — smart aggregation of smaller terms into conceptually meaningful statistically significant clusters, adjusting bids to capitalize on time-of-day and day-of-week patterns, etc etc etc. — but balancing costs and benefits is the key idea.

When a user clicks on several ads from the same advertiser, most folks credit the order to the last click before it.

Why is this last-click-gets-credit assumption important?

Because the effectiveness of PPC ads varies a lot. (Measuring effectiveness by, say, sales-per-click.) Some ads are great. Many aren’t. Real stinker ads are quickly culled, of course. Even then, there’s still large variability in ad effectiveness within a PPC keyword portfolio.

There’s a much talk online and at conferences about understanding the value of ads on the advertiser’s own brand. (Example: Land’s End advertising on “Lands End“) Ditto on ads on broad generic terms (Example: Land’s End advertising on “parka“, for example.) Very important ideas.

So ads (clicks) vary in effectiveness. And we all aim to buy more of what works and less of what doesn’t. All well and good. Here’s the catch — clicks aren’t independent. Clicks from the same user in a reasonable time window comprise a clickstream.

We wondered: how would our economic analyses change when we considered clickstreams vs. considering clicks? Could we build a mathematical model to understand how earlier clicks in the the clickstream lay the groundwork for better or worse clicks to come?

Let’s make this concrete with an example. Let’s assume the generic broad term “parkas” for a hypothetical retailer had terrible economics. OK. But suppose “parkas” is frequently followed by additional searches, like “boys parka” or “ski parka” or “gortex parka”, and these more detailed phrases have great conversion. If the (poor) “parka” ad often sets the stage for (great) “gortex parka” ad, shouldn’t “parka” receive some of the credit?

Like the soccer player who seldom scores but sets up many assists, there could be value in bidding more on broader terms higher up in the conversion funnel, even if their stand-alone economics didn’t warrant it, if we could prove such terms set the stage for later winning searches.

Could we build such a mathematical model?

Yup. And we did, calibrating it a random sample of half million Google and Yahoo PPC clickstreams from 2005.

What we learned suprised us.

We found that crediting the order to the last ad before it turns out to be a pretty good approximation. In other words, not all that much benefit passes back up the clickstream to earlier phrases.

Going back to the soccer analogy, our data showed that many paid search campaigns resemble teams of great players (score many goals) and average players (who fewer goals) — but not that many great assist players.

Interesting.

To us, this casts some doubts on the “conversion funnel” approach to paid search. “Spend heavily on ‘parka’s”, pundits might advise, “even if the stand-alone results for ‘parkas’ aren’t all that great, as these broad generic terms reflect customers on the top of the conversion funnel. ‘Parka’ might not be great, but it sets the stage for great ads like ‘gortex parka’, so the high volume generic terms are worth it, despite their numbers.”

Can we trust our model? I think so.

The model isn’t perfect. There minor technical quibbles, like the one-step Markovian assumption.

The big gaping potentential imperfection? This study did not take into account sales spillover into call centers and stores.
This is a significant concern, and one raised by Yahoo’s Diane Rinaldo on the
SES panel . Going back to the “parkas” example: maybe term doesn’t look so hot online, but maybe all those searchers call the call center or go into the store and buy there.

Sure. Could be.

However, our sample included both internet pure-plays, catalogers, and national store retailers — and our findings didn’t vary by type. We had expected to see much more value pass back up the clickstream to earlier ads, and we didn’t see it in our data.

There’s nothing wrong with buying generic words for branding, and they may indeed drive store and call-center traffic.

But for the retailer concerned with using online ads driving online sales, when the pundits preach spending more on poorly-performing high-volume terms because they “play an important role on the top of the conversion funnel”, we’d respond “maybe, maybe not.” Proceed with care. Track and test as you move ahead.

Here’s the link to the study: Click Streams, Complexity, and Contribution: Modeling Searcher Behavior Using Markov Models.

Rumor has it the major engines will be giving advertisers more insight into their paid clickstreams as 2006 progresses. Stay tuned.