Bing’s Broad Match Problem
Yahoo is set to announce its Q1 earnings later today and unlike for Google, expectations are understandably low. Earlier this year, we pointed out some troubling trends in Search Alliance traffic and we offered some anecdotal evidence of the Bing powered ad listings doing a poor job monetizing their SERPs. We’ve had some time to review the data and we can offer a bit more rigorous analysis of one aspect of Bing ad serving that appears to be weighing down Alliance revenues: broad matching.
We are most concerned with how the engines are matching keywords to search queries, not the matchtype settings of the keywords themselves, but to ensure a fair assessment, we reviewed only clients with similarly managed programs on the Alliance and Google (i.e those with nearly identical term lists and similar matchtype usage).
To account for nuances in the engines’ matchtype definitions, for our purposes, we will say an exact match has been made if the keyword and the search query are identical. If the entire keyword can be found in the search query in one piece, we’ll consider that a phrase match and everything else is a broad match. Given those definitions, here’s what we find.
Broad Matching Rates
Google has a 34% higher broad match rate than Bing, as defined by the percentage of broad matched clicks over all clicks for each. At the same time, Bing relies upon phrase match (as defined above) nearly twice as much as Google. The two engines have more similar rates of exact match usage with Bing coming in 10% higher than Google there.
This suggests Bing is having difficulty assessing when a broad match click will be valuable to advertisers and it may explain a large part of the perceived improvement in the combined Bing and Yahoo sales per click since their alliance. All else being equal, the more closely a keyword matches a search query the better it will perform, but more broadly matched traffic still holds value.
It appears that Bing’s ad serving is cutting broad traffic relative to when Yahoo provided its own ads and, although the value of an Alliance click has increased overall, we have not been able to keep traffic levels up even as we have increased bids.
Reviewing our pre-Alliance data, we find that Yahoo was by far the most liberal in its use of “Advanced” ad matching with roughly 60% of all clicks meeting our definition of broad match above, although Yahoo’s often vexing keyword canonicalization may skew those numbers a bit. At the time we found Yahoo traffic to be 15% less valuable than Bing’s for the same queries, so that wasn’t necessarily ideal either.
Keyword Tokens in Search Query
To get a better idea of where Bing may be falling short, we refined the data down to only those queries where a broad match was made to a keyword containing exactly three “tokens”, or words within the keyword phrase. In these instances we found that Bing was 27% more likely than Google to require that all keyword tokens were contained in the search query exactly as they appear in the keyword (singulars/plurals, etc. were considered different tokens).
Along with its significantly higher share of phrase match clicks, this shows that Bing has a much more literal view of matching keywords to queries than Google. In other words, not only does Bing make a broad match to a keyword less frequently than Google, but even when it does, it is far more strict. In a way, Bing’s broad match works more like Google’s broad match modifiers than Google’s own broad match.
While we’ve certainly complained in the past about how expansive Google’s broad match can be, we can always put a damper on it through negative keywords, smart bidding and other tactics. What we cannot easily do is efficiently reach more eyeballs searching unique queries with an already very well built-out keyword list.
Traffic by Keyword Tokens
Lastly, we compared Alliance and Google traffic levels by the number of tokens in the keyword phrase shown for all match types. What we find is that Bing traffic levels match up to Google’s far better for shorter keywords and then fall off steeply as we move to longer keyword phrases.
This does not appear to be the result of Bing showing one token keywords more often for longer search queries, as we find that Bing and Google show one word keywords at nearly identical exact match rates. Instead, it appears that Bing has greater difficulty matching or chooses not to run ads as often for longer search queries.
We don’t mean to dwell on the woes of Bing and Yahoo, but it’s clear that the road has been bumpier than many had hoped for their Alliance. In Q4 2010, Yahoo reported an 18% decline in Search revenue and we’ve seen analysts predicting a similar drop for Q1 2011. Yahoo CEO Carol Bartz has pointed to the second half of 2011 as the time-frame for improved results and we hope that’s the case as it will be a win-win for Bing and Yahoo and for marketers like ourselves.
We do know that the folks at both Bing and Yahoo are working very hard to make that turnaround a reality and recent natural search share reports have been promising. So, let’s not count them out yet.