Oct 192009

PPC Engine Loyalty Study

How loyal are people to their search engines?

This is one of those questions that has crossed my mind a number of times, but I never got around to looking at the data until now. I have no doubt that there are comScore studies out there, but with all due respect to those sharp folks, I don't think their sample is representative of the general population.


We took a random sample of our clients, and studied just over 1.8 million orders placed through our ads on those clients' sites. We wanted to find out how loyal those buyers were to the engines they used. We eliminated all clicks from content networks, comparison shopping engines and other programs, so that we were left with only Google's Adwords Network, Yahoo's Network, MSN/Bing, and Ask.com's Network. We also eliminated clicks on ads for different clients, and clicks more than 60 days prior to the order.

With this streamlined data set we answered the question:

For each engine, what fraction of the people using that engine first used that engine exclusively during their browsing?

We also looked to see whether the data looks different before Bing launched and subsequent to its launch.


At first blush, the engines all appear to have fairly loyal followings.

This says that Google enjoys the highest level of loyalty with 97.6% of buyers who first touched a Google ad using only Google ads prior to their purchase.

However, this probably isn't the right way to think about the phenomena. As we've noted previously, most PPC buyers touch only one ad before buying. The fact that they used only that one engine is perhaps less compelling evidence of 'loyalty' than we would see if we take a different tack.

In this next pass, we eliminated all the orders placed by people who only clicked one ad one time. What's left answers the question:

For people who clicked more than one ad before buying, what fraction used the same engine/network for each click? And, has this changed since Bing launched?

These data paint a much more dramatic picture.

  • Pretty clearly, Google users are much more loyal to their engine than the users of Yahoo and Bing.
  • The degree of loyalty hasn't shifted much since Bing's launch, and one might surmise that Google folks are the ones testing the waters a bit as their loyalty number seems to have dipped a tiny bit.
  • The Ask.com numbers are distorted by two factors: 1) on Ask.com Google ads intermix with their own; and 2) Most Ask.com traffic comes from their network, not Ask.com. As such, I'm eliminating them from the rest of the discussion (you'll note some of the percentages below won't quite add up to 100% as a result of excluding them after running the numbers.)

Once they leave, where do they go?

To do this analysis correctly, one would need to do a Markov chain analysis or something like it to look at transition probabilities. I didn't have time to do that. Instead, I just looked at a gross measure of first engines and last engines within the set of multiple ad touch buyers. This obviates the distinction between Martha who went from Google => Yahoo => Google, and Ben who uses only Google. As such, having first engine and last engine the same is not identical to being an engine loyalist as studied above.

This is pretty cool data even given the crudeness of the study.

At first blush one might conclude that Yahoo and MSN are in big trouble: not only are Google users more loyal, but users of other engines are much more likely to move to Google than to any other. However, percentages lie. Google is the biggest so it's normal that, for the subset that use more than one engine starting with MSN/Bing, most will go to Google.

The more interesting question is:

Do they migrate one way or another disproportionately to the size of the market share?

That's what we show below. The expected percentage is calculated taking the relative market share -- as defined by the entire data set from the first pass -- of the other engines into account. We then calculated the percentage difference from the expected values.

This is kind of interesting! What strikes me is that MSN/Bing users are disproportionately more likely to move to Yahoo than to Google, and Yahoo users are disproportionately more likely to move to MSN/Bing. Perhaps this indicates that these folks really are committed to finding a Google alternative? I don't know. I'm not entirely confident of my methodology/thought-process, and given that the first - last view is a pretty hacky approach we probably shouldn't read too much into it.


Google appears to have by far the "stickiest" user base. This suggests that Bing has its work cut out for it if it is to gain share. That said, those who don't use Google primarily are more likely to prefer other alternatives to Google than to follow the majority. The market demands a viable alternative.


This may be of most interest to the engines themselves -- though they certainly already know this stuff -- and perhaps to my friends at the big investment banks who want to gauge the future performance of those companies.

From a PPC management perspective, it's kind of a non-issue. The lion's share of the traffic comes from one touch buyers and engine loyalists. Moreover, by raw order count, the number of people who move from a first ad touch on engine X to a last ad touch on engine Y is almost exactly equal to the number moving from Y to X. This is true for all of the interaction Google <=> Yahoo; Yahoo <=> Bing; Bing <=> Google. As such the net effect on ad buying strategy is nil.


Almost too many to list, but here are three more that might not be obvious.

  • We're only counting clicks on the advertisements. If someone clicks on a Google ad, then searches on Bing and clicks on a natural search link from that advertiser and makes a purchase, we're calling them a Google-only user.
  • Engine loyalty may have nothing to do with satisfaction levels. It is entirely possible that Yahoo and Bing are guaranteed to have lower loyalty numbers simply because of Google's dominant market share. "Everyone" uses Google, so the curiosity-factor for Yahoo and Bing users has to be greater.
  • We're looking at Networks, here, so when Ebay switches from Yahoo's network to Google's people who used Ebay ads exclusively appear to have switched loyalties. Also, people who use off-beat search engines and bounce around might seem to be either loyalists if the sites all happen to be in one network, or switchers if they aren't in the same network, even though their behavior is essentially the same.

I'd love to hear what others think about this. Am I looking at the numbers the right way? Am I thinking about the numbers the right way. Does this actually tell a story?


12 Responses to "PPC Engine Loyalty Study"
Very similar results to what I see in my Multichannel Forensics projects, good job!
That's very reassuring, Kevin. I'm sure you took the time to do the math right, too :-)
Very impressed with the data and analysis here. It could perhaps be used when thinking about keeping ad copy consistent between engines: Google ads can do their own thing, but Bing and Yahoo should carry a similar message since they are more likely to be seen by the same people. Or perhaps you want to make them different because they are seen by the same people. Would doing the transition probability stuff tell you more or would it just give you more confidence in the results you have already?
Thanks, Richard! Doing the transition probability calculations would just give us more accurate percentages to capture the true flow. I wouldn't expect those more accurate numbers to be very different from what's presented here. The fact that Kevin Hillstrom sees the same basic numbers gives me confidence that this hack approach gets pretty close to the truth. Given that the vast majority of traffic is single touch or engine loyal (that first table), I think you're best served trusting your copy tests on each engine. Writing copy for the thin slice coming from other engines at the expense of what works for the majority on that engine might be count-productive.
Social comments and analytics for this post... This post was mentioned on Twitter by HJSNews: PPC Engine Loyalty Study: A new study on search engine loyalty from the RKG skunk-works http://bit.ly/yY3Eq...


Check out what others are saying...
[...] PPC Engine Loyalty Study, Rimm Kaufman [...]
[...] PPC Engine Loyalty Study Interesting study about how people search on one engine and may click on an ad from another one before final purchase. I think your main takeaway should be to answer the question, "Should I be on multiple engines?". Many advertises only use Google as that's the biggest engine out there. However, if you only use on source of advertising, you will miss some potential customers. (tags: ppc) [...]
[...] PPC Engine Loyalty Study19 Oct 2009 by George Michie  A new study on search engine loyalty from the RKG skunk-works. – [ ] [...]
[...] feed for updates on this topic. // The Rimm-Kaufman Group, as usual, has just posted an amazing analysis of over 1.8 million orders placed through their clients’ sites. The premise was simple - they wanted to find out how loyal [...]
[...] grupa objavila je zanimljivu studiju o utjecaju i povjerenju korisnika naspram tražilica i njihovih oglasnih sustava, [...]
[...] Rimm-Kaufman Group published an interesting study on the impact and trust given by users with respect to search engines and their advertising sy..., and the way in which they react to advertising systems that publish their advertisements. Even [...]

Leave A Comment