THE RKGBLOG

Bing’s Broad Match Problem

Yahoo is set to announce its Q1 earnings later today and unlike for Google, expectations are understandably low.  Earlier this year, we pointed out some troubling trends in Search Alliance traffic and we offered some anecdotal evidence of the Bing powered ad listings doing a poor job monetizing their SERPs.  We’ve had some time to review the data and we can offer a bit more rigorous analysis of one aspect of Bing ad serving that appears to be weighing down Alliance revenues: broad matching.

We are most concerned with how the engines are matching keywords to search queries, not the matchtype settings of the keywords themselves, but to ensure a fair assessment, we reviewed only clients with similarly managed programs on the Alliance and Google (i.e those with nearly identical term lists and similar matchtype usage).

To account for nuances in the engines’ matchtype definitions, for our purposes, we will say an exact match has been made if the keyword and the search query are identical.   If the entire keyword can be found in the search query in one piece, we’ll consider that a phrase match and everything else is a broad match.  Given those definitions, here’s what we find.

Broad Matching Rates

Google has a 34% higher broad match rate than Bing, as defined by the percentage of broad matched clicks over all clicks for each.  At the same time, Bing relies upon phrase match (as defined above) nearly twice as much as Google.  The two engines have more similar rates of exact match usage with Bing coming in 10% higher than Google there.

This suggests Bing is having difficulty assessing when a broad match click will be valuable to advertisers and it may explain a large part of the perceived improvement in the combined Bing and Yahoo sales per click since their alliance.  All else being equal, the more closely a keyword matches a search query the better it will perform, but more broadly matched traffic still holds value.

It appears that Bing’s ad serving is cutting broad traffic relative to when Yahoo provided its own ads and, although the value of an Alliance click has increased overall, we have not been able to keep traffic levels up even as we have increased bids.

Reviewing our pre-Alliance data, we find that Yahoo was by far the most liberal in its use of “Advanced” ad matching with roughly 60% of all clicks meeting our definition of broad match above, although Yahoo’s often vexing keyword canonicalization may skew those numbers a bit.  At the time we found Yahoo traffic to be 15% less valuable than Bing’s for the same queries, so that wasn’t necessarily ideal either.

Keyword Tokens in Search Query

To get a better idea of where Bing may be falling short, we refined the data down to only those queries where a broad match was made to a keyword containing exactly three “tokens”, or words within the keyword phrase.  In these instances we found that Bing was 27% more likely than Google to require that all keyword tokens were contained in the search query exactly as they appear in the keyword (singulars/plurals, etc. were considered different tokens).

Along with its significantly higher share of phrase match clicks, this shows that Bing has a much more literal view of matching keywords to queries than Google.  In other words, not only does Bing make a broad match to a keyword less frequently than Google, but even when it does, it is far more strict.  In a way, Bing’s broad match works more like Google’s broad match modifiers than Google’s own broad match.

While we’ve certainly complained in the past about how expansive Google’s broad match can be, we can always put a damper on it through negative keywords, smart bidding and other tactics.  What we cannot easily do is efficiently reach more eyeballs searching unique queries with an already very well built-out keyword list.

Traffic by Keyword Tokens

Lastly, we compared Alliance and Google traffic levels by the number of tokens in the keyword phrase shown for all match types.  What we find is that Bing traffic levels match up to Google’s far better for shorter keywords and then fall off steeply as we move to longer keyword phrases.

This does not appear to be the result of Bing showing one token keywords more often for longer search queries, as we find that Bing and Google show one word keywords at nearly identical exact match rates.  Instead, it appears that Bing has greater difficulty matching or chooses not to run ads as often for longer search queries.

We don’t mean to dwell on the woes of Bing and Yahoo, but it’s clear that the road has been bumpier than many had hoped for their Alliance.  In Q4 2010, Yahoo reported an 18% decline in Search revenue and we’ve seen analysts predicting a similar drop for Q1 2011.  Yahoo CEO Carol Bartz has pointed to the second half of 2011 as the time-frame for improved results and we hope that’s the case as it will be a win-win for Bing and Yahoo and for marketers like ourselves.

We do know that the folks at both Bing and Yahoo are working very hard to make that turnaround a reality and recent natural search share reports have been promising.  So, let’s not count them out yet.

Technorati Tags: ,

  • Mark Ballard
    Mark Ballard is Director of Research at RKG.
  • Comments
    8 Responses to “Bing’s Broad Match Problem”
    1. Chip Jones says:

      Thank you for the research! We have also found peculiar behavior when it comes to the way that Bing matches the actual search phrases to the adgroups/keywords we’ve setup in our campaigns. It’ will be interesting to see how this all shakes out…

    2. Mark Ballard Mark Ballard says:

      Thanks for corroborating Chip! On their earnings call yesterday, Yahoo! all but threw Microsoft under the bus for their troubles in search, so the pressure is on.

    3. Matthew Mierzejewski Matthew says:

      Mark,

      “…Bartz said technical issues with Microsoft’s adCenter platform are limiting the number times users click on search ads on Yahoo’s pages”
      http://online.wsj.com/article/BT-CO-20110419-715151.html

      Thrown under the bus is right! It would appear the differences in Y! and MSN broad match algorithms HAS resulted in a loss in revenue. Your research puts the data behind Carol’s words.

    Trackbacks
    Check out what others are saying...
    1. [...] Bing’s Broad Match Problem, Rimm Kaufman [...]

    2. [...] We’re watching with great interest to see how it affects your CTR. Bing, on the other hand,  has some broad match issues that are negatively affecting traffic. Mark Ballard at Rimm-Kaufman looks at how Bing has difficulty determining when a broad match click [...]

    3. [...] Bing’s ad serving is much more restrictive than Yahoo’s was before the Alliance, as shown by a sharp drop in broad matched traffic. [...]

    4. [...] Earlier this month when the adCenter Blog confirmed that they were making changes to Bing’s broad and phrase match behavior, I was optimistic about the prospects. In a few recent posts, I’ve touched on some issues that may be contributing to the relatively weak financials Microsoft and Yahoo have reported for their respective paid search efforts. One of the larger issues appeared to be adCenter’s overly restrictive ad serving. [...]

    5. [...] RKG looked at this last year, we found adCenter was broad matching our ads to queries just 40% of the time for Yahoo, [...]