How loyal are people to their search engines?

This is one of those questions that has crossed my mind a number of times, but I never got around to looking at the data until now. I have no doubt that there are comScore studies out there, but with all due respect to those sharp folks, I don’t think their sample is representative of the general population.

METHODOLOGY

We took a random sample of our clients, and studied just over 1.8 million orders placed through our ads on those clients’ sites. We wanted to find out how loyal those buyers were to the engines they used. We eliminated all clicks from content networks, comparison shopping engines and other programs, so that we were left with only Google’s Adwords Network, Yahoo’s Network, MSN/Bing, and Ask.com’s Network. We also eliminated clicks on ads for different clients, and clicks more than 60 days prior to the order.

With this streamlined data set we answered the question:

For each engine, what fraction of the people using that engine first used that engine exclusively during their browsing?

We also looked to see whether the data looks different before Bing launched and subsequent to its launch.

FINDINGS

At first blush, the engines all appear to have fairly loyal followings.

This says that Google enjoys the highest level of loyalty with 97.6% of buyers who first touched a Google ad using only Google ads prior to their purchase.

However, this probably isn’t the right way to think about the phenomena. As we’ve noted previously, most PPC buyers touch only one ad before buying. The fact that they used only that one engine is perhaps less compelling evidence of ‘loyalty’ than we would see if we take a different tack.

In this next pass, we eliminated all the orders placed by people who only clicked one ad one time. What’s left answers the question:

For people who clicked more than one ad before buying, what fraction used the same engine/network for each click? And, has this changed since Bing launched?

These data paint a much more dramatic picture.

• Pretty clearly, Google users are much more loyal to their engine than the users of Yahoo and Bing.
• The degree of loyalty hasn’t shifted much since Bing’s launch, and one might surmise that Google folks are the ones testing the waters a bit as their loyalty number seems to have dipped a tiny bit.
• The Ask.com numbers are distorted by two factors: 1) on Ask.com Google ads intermix with their own; and 2) Most Ask.com traffic comes from their network, not Ask.com. As such, I’m eliminating them from the rest of the discussion (you’ll note some of the percentages below won’t quite add up to 100% as a result of excluding them after running the numbers.)

Once they leave, where do they go?

To do this analysis correctly, one would need to do a Markov chain analysis or something like it to look at transition probabilities. I didn’t have time to do that. Instead, I just looked at a gross measure of first engines and last engines within the set of multiple ad touch buyers. This obviates the distinction between Martha who went from Google => Yahoo => Google, and Ben who uses only Google. As such, having first engine and last engine the same is not identical to being an engine loyalist as studied above.

This is pretty cool data even given the crudeness of the study.

At first blush one might conclude that Yahoo and MSN are in big trouble: not only are Google users more loyal, but users of other engines are much more likely to move to Google than to any other. However, percentages lie. Google is the biggest so it’s normal that, for the subset that use more than one engine starting with MSN/Bing, most will go to Google.

The more interesting question is:

Do they migrate one way or another disproportionately to the size of the market share?

That’s what we show below. The expected percentage is calculated taking the relative market share — as defined by the entire data set from the first pass — of the other engines into account. We then calculated the percentage difference from the expected values.

This is kind of interesting! What strikes me is that MSN/Bing users are disproportionately more likely to move to Yahoo than to Google, and Yahoo users are disproportionately more likely to move to MSN/Bing. Perhaps this indicates that these folks really are committed to finding a Google alternative? I don’t know. I’m not entirely confident of my methodology/thought-process, and given that the first – last view is a pretty hacky approach we probably shouldn’t read too much into it.

CONCLUSIONS

Google appears to have by far the “stickiest” user base. This suggests that Bing has its work cut out for it if it is to gain share. That said, those who don’t use Google primarily are more likely to prefer other alternatives to Google than to follow the majority. The market demands a viable alternative.

WHO CARES?

This may be of most interest to the engines themselves — though they certainly already know this stuff — and perhaps to my friends at the big investment banks who want to gauge the future performance of those companies.

From a PPC management perspective, it’s kind of a non-issue. The lion’s share of the traffic comes from one touch buyers and engine loyalists. Moreover, by raw order count, the number of people who move from a first ad touch on engine X to a last ad touch on engine Y is almost exactly equal to the number moving from Y to X. This is true for all of the interaction Google <=> Yahoo; Yahoo <=> Bing; Bing <=> Google. As such the net effect on ad buying strategy is nil.

LIMITATIONS

Almost too many to list, but here are three more that might not be obvious.

• We’re only counting clicks on the advertisements. If someone clicks on a Google ad, then searches on Bing and clicks on a natural search link from that advertiser and makes a purchase, we’re calling them a Google-only user.
• Engine loyalty may have nothing to do with satisfaction levels. It is entirely possible that Yahoo and Bing are guaranteed to have lower loyalty numbers simply because of Google’s dominant market share. “Everyone” uses Google, so the curiosity-factor for Yahoo and Bing users has to be greater.
• We’re looking at Networks, here, so when Ebay switches from Yahoo’s network to Google’s people who used Ebay ads exclusively appear to have switched loyalties. Also, people who use off-beat search engines and bounce around might seem to be either loyalists if the sites all happen to be in one network, or switchers if they aren’t in the same network, even though their behavior is essentially the same.

I’d love to hear what others think about this. Am I looking at the numbers the right way? Am I thinking about the numbers the right way. Does this actually tell a story?

Technorati Tags: Bing Market share, MSN Yahoo agreement, search engine market share