Jan 212009

Google Broad Match: A Change for the Worse

If you monitor results closely as we do, you may have noticed a decline in the performance of Google's Broad Match over the last 5 months or so. We're having to lower bids on broad-matched keywords that used to work well.

We assumed that Google was simply playing with "the dials" again balancing their revenue against ad quality, but just recently we found what we think is actually a bug in Google's system. We pointed out a behavior to them that they had assured us didn't exist, and when confronted with the evidence they said "we certainly didn't intend for it to work that way."

Let's start from the beginning:

Broad match, which became "Advanced match" a few years back, can be enormously helpful. We hear that in any given month something like 25% of user searches are unique -- never-seen before. Anyone who searches through log files to compare user searches to the advertised phrase has seen this kind of thing: "I want to buy a widget, preferably red, but I don't know where to find one". As methodical and obsessive as we are about keyword creation, we probably don't have that one. People type in all kinds of crazy permutations on makes, model numbers, skus with and without a color preference with/without spaces, with/without typos that make a perfectly comprehensive list impossible.

We know that the broad matched traffic doesn't convert as well as traffic that exactly matches the ad phrase, but creating separate broad match and exact match versions of keywords and advanced atomic bidding allow us to capture the best of both worlds.

However, the truly troubling behavior of broad match that we've seen recently occurs when a broad matched ad is served instead of an exact matched ad in your own account. We understand that for the crazy user search above, Google might choose to serve either our broad matched ad Keyword = "widget", or our broad matched ad Keyword = "Red Widget". We understand that Google will probably serve the one of those two ads that makes them the most money (CPC * CTR).

What we didn't like was that sometimes a person searching for "Red Widget" would be served a "Widget" ad, when we have a perfectly good "Red Widget" ad ready to go. The Red Widget ad is more relevant, has a better landing page and will provide a better user experience than the "Widget" ad. We feared that broad matched ads in effect competed with each other, with the highest bid * CTR winning out over dead-on matches.

Google assured us that this self-competition did not exist. They told us that if you have a keyword in the account that exactly matches the user search only that ad and its bid will be taken into account, and broad matched variants will not be served even if they have higher bids. They said there are a few rare cases involving ads for the same keywords in different campaigns where there could be glitches, but that in almost every case they've investigated the situation only occurred when the exact matched ad had been halted by campaign budgets.

That got us thinking...

If self-competition does happen when the keyword that matches exactly is off because of budgets, does that also mean that keywords that are paused are also subject to such cannibalism by other broad matched ads? Say for example that you used to carry "steel widgets" but you no longer do. Simply pausing the keyword will result in people who were served the "steel widgets" ad, now being served the "widgets" ad, or the "blue widgets" ad, or the "wooden widgets" ad -- whichever makes Google the most money. You wanted to eliminate the traffic, but instead, you've shifted the traffic over to other keywords which are potentially less targeted and more expensive!

We asked Google's "Product Management" team for broad match and they confirmed that pausing ads in the account does, in effect, make those phrases available for broad matching to other ads. Google said that if an ad isn't active, then the broad match algorithm ignores it and seeks to make the best match it can.

That got us thinking...

What if an ad is not paused, but simply not bid high enough to be "active" -- did that effectively blind Broad match to the existence of the ad and allow this self-competition/cannibalization? We could deal with the "pause" problem by simply bidding a penny on ads we'd rather pause, unless bidding too little was effectively the same as pausing.

Google agreed that that used to be the behavior "However," they said "that's no longer a problem because we've done away with the notion of minimum bids!"

That got us thinking:

"Hey guys, what we think happened is this: Google did away with minimum bids to be active but replaced that concept with a minimum bid to be on the first page. We think what happened is that as of August/September, if an ad falls below the minimum first page bid it becomes "inactive" for search in broad match and other ads that do meet that minimum are shown in its place."

Google was silent for a minute and then said: "That shouldn't be the case. That isn't supposed to be how it works. Can you prove that that's what's happening?"

We could, and did.

What happens is this: Keyword "Widget" performs only so-so. Doesn't matter whether it's on broad match or exact match or if you have two versions. The competitive landscape is such that others can afford to spend more -- or can't but position bidding systems force them to overspend. "Widget" drops off the first page and immediately your "Blue Widget" ad (broad matched) starts showing since it converts better and has a higher bid than "Widget". The poor traffic that used to go to "Widget" now effectively contaminates the data for "Blue widget". Indeed, the traffic that converted poorly when dropped on the "widget" page will convert even worse when it lands on the "blue widget" page instead. The "blue widget" ad is bid down the page until it drops off the page, when your "Red Widget" ad starts firing...and so on. A cascade failure of broad matched performance because your targeted ads start competing with each other over the lousy traffic that you can't afford to spend money on.

What we saw as a result of this apparently inadvertent change in Google's logic in August was keywords that used to work fine on broad match no longer worked at all. We had to drop bids because of this which reduced Google's ad revenue and some of our clients' sales as well.

Using poor performing keywords as exact match negatives across all campaigns is one strategy to deal with this, but does "poor performing" mean any keyword that drops off the first page?!? The bidding landscape is so dynamic that what was off the first page yesterday might be back on today -- do we really need to add and delete exact match negatives based on this kind of fluctuation?!? It really makes it impossible to manage broad match intelligently. We lose the good stuff along with the traffic we don't want.

Bad for advertisers, bad for users, bad for Google.

We're very encouraged that Google sees this as a problem, too. Google is not evil. Indeed, Google is better behaved than any company in history that has as much power as it does. However, Google is bifurcated. There is a strong contingent of engineers and account reps who want to provide the best possible experience to their users, and the most control possible to advertisers to manage their advertising spend as effectively as they can. There is another contingent within Google who, like any for-profit business, wants to make as much money as possible. Sometimes these groups are at tension with each other.

We believe that improving the broad-match controls available to advertisers will benefit all parties concerned. We made a number of suggestions to the product managers -- I'll detail these in another post -- and those were very well received. Many folks at Google have the right instincts on these issues. We just hope they prevail in the internal debates to come.

I don't know whether Google will tell us if they find that we're right. I also don't know if they'll fix the problem -- we suggested that they simply prevent broad match from competing with any keyword in the account, regardless of bid, paused or not paused. That would represent a pretty big change, and they may have concerns about the impact on their bottom line.

Hopefully, they'll fix the problem and let us know when they've done so. When that happens broad match should work better for all of us.

We'll let you know if we hear more!


32 Responses to "Google Broad Match: A Change for the Worse"
One of our clients pointed out a similar thing a couple of months ago. It was a bit of a pain to add negative keywords to each ad group to prevent ads from the wrong ad group showing. Glad to hear that Google don't mean this to happen and that it might get sorted out. Thanks for mentioning this to them.
James Zolman says:
We have reports and data that correlate quite well with what you are describing in this post. I really appreciate how you described the problem! Sphunn: http://sphinn.com/story/97928
Richard and James, thanks for the feedback. Hopefully, if all the agencies and advertisers who watch their numbers carefully put pressure on Google, we can get them to build in the controls we need to manage accounts intelligently. As I pointed out to Google, if an athelete doesn't know the rules of a game, or sprained an ankle the last time they played, they're likely to play tentatively and conservatively. Once they become more sure of themselves they can play more aggressively. More money for Google, and for the advertisers.
Very interesting - thanks a lot for explaining. The interaction between match types on same or different keywords is one of those realities that is incredibly important but very difficult to see. I'm going to see if I can see any queries which used to gain traffic on one keyword and now get it from another keyword - because of the effect you describe.
Great post, George! It's incredible that such an arcane aspect of AdWords can - and IMO probably does - have such a large effect on the marketplace. If someone ever starts an SEM Ph.D conference series, you should be the keynote speaker.
Very kind of you to say, Chris, thank you. I don't know if it's related, but from our data it looks like Google shed a wee bit of market-share back to Yahoo since August (81% down to 78 or 79%). Yahoo stopped serving Google ads at some point, but that shouldn't have swung the bar that much. Could it be that dropping bids on broad matched ads actually cost Google in a pretty significant way? Who knows! George
Thanks, George for doing such a great job of explaining in detail some of the things that we are also seeing. The part about the inactive, active, minimum bid, first page bid, etc. really struck a chord. Hadn't put that piece of the puzzle together until your article today.
Great post and thoroughly explains one of the many issues with Broad Match. I've been very vocal with our Google team concerning our issues with Broad, including asking for an opt-out of what they call "expanded broad match." (We find Advanced, or "expanded" broad match really hurts ROI if the site is focused on brands.) Thanks for the post on a rarely discussed topic.
Thanks Craig, Shelley and Bobby. Bobby we're with you. We argued all along that the old broad match should be a fourth match-type choice for those of us who want a bit more control. I'm writing a follow-up piece for Search Engine Land that will discuss some of the other controls we'd like to see. Probably out next Monday or Tuesday.
Craig, Shelley, Bobby -- Here's a link to the SEL post George mentioned just above: searchengineland.com/heads-up-google-broad-match-controls-we-need-16307
Hi! We have also seen this behavior, but not known the source to the problem, thank you indeed for taking this up with google. What we have had to do was in som cases to put the keywords "red widget" and "widget" in separate ad groups and then ad the match type "exact match negative" :) -[widget] to the red widget group. tedious but it works! Ludvik
Brad Geddes says:
Thanks for this post. I was assuming it was happening, but hadn't had a chance to dig into the data yet - you saved me a lot of time :) Keep up the nice work - love this blog, brad
Ryan Pryor says:
RKG (George specifically this time)... you guys rock so hard!! This really helps give me insight into why some of my accounts are not even operating like I'd normally expect. This appears to be (one of) the reason(s) why an account I just overhauled and relaunched isn't responding hardly at all. One frustrating thing I've noticed, too, is that Google makes 1st page bid suggestions, but won't always let you set your KW bids at those levels. I've been running into multiple KWs in fresh ad groups being rejected from Editor posting due to "the KW CPC bid is either too high or too low." Well, no, Google, it's where you said it needed to be set to even function... So, I delete the bid and it posts after all. Weird. SO, in light of these lovely RKG findings... should we stick to PH and EX matched KWs until Google can figure itself out? My clients don't tend to have budgets large enough to allow this kind of potential mass destruction to their accounts/bids. Thanks, George & team, ryan
Hi Brad and Ryan, thanks for the kind words. Right now, no one in the retail sector can really afford this bug. What we're doing to apply a tourniquet is maintaining separate EM and BM campaigns, bidding less on the BM versions. Adding as exact match negatives any KW in the account that have been paused or deleted, and finding any general KW that would be high traffic, that are well off the first page and throwing them in as negatives as well. Colossal pain, borderline unmanageable, but it seems the best we can do for now. George
Darren Monk says:
We have had exactly this issue. On researching keywords in a new add we spent hundreds of pounds on a Phrase Match key word "shower tray" we have spent months fine tuning lots of other shower tray add groups with phrase and exact match keywords. Example [large shower tray] & "large shower tray" are in there own dedicated add group. In the testing adverts for "shower tray" we picked up lots of traffic for many of our other fine tuned adds including [large shower tray]. On speaking with the guys at Google they explained it was because of two thing 1, quality score and 2, bid price. When I explain that our QS was "Great" for both adds so Google must then be directing traffic to where it will make the most money (highest bid) I was told no this is not the case and it is not a bug in their system (I never said it was at this point). I went on to say "why then do you send random traffic to a phrase match when I have spent lots of time and money on fine-tuning our adds (as they tell us to do) and carefully choose landing pages so that (which makes Goole look good because they can provide exactly what people are looking for in any given search) do you push traffic to where you will receive the most money". I was told that they don't and that it was to do with QS again. Our QS was the same (Great) for both adds. The guy said that there is a closer QS that has more detail in the system. The trouble is I cant see it and neither can they. It a built in QS? The bottom line is Google has all of us by the balls (sorry ladies) and there is not a thing we can do about it. The best advise they could give was, start building long & time consuming negative keyword lists and add them to all adverts where this could happen. THANKS GUYS
Tom Hale says:
Great piece. Might explain some of the things I have been seeing. I use less broad match and dynamic insertion because I trust Google's "automation" less and less. Seems to me the bean counters are winning out over the relevancy evangelists. Tom Hale AdWords Specialist Thomas Creek Concepts
Ryan Pryor says:
George, In your proposed BM vs EM campaign structure, designed to "combat" the Google BM KW cannibalization, do you have to construct those as all new campaigns, and pause your originals? What I mean is, do you have to reconstruct your approach in older accounts, and thus lose your KW history & QScore legacy...? Seems like that would be a bit of an issue... but maybe an acceptable loss considering the size of the problem. Ryan
Thanks for your comments Darren and Tom. Ryan, our reps have explained to us that KW Quality Score is based on the user reaction to the KW combined with copy blocks. So, as long as the new campaigns make use of the same copy as the old, the KW QS should be unaffected. AdGroup and Campaign QS reestablishes itself quickly because the volume of traffic is so much greater than on any single KW. As you suggest, any blip in the QS should be greatly outweighed by the benefits of better targeting bids.
huayin says:
Thanks for the post. The problem looks more interesting if taken a perspective of google ad-match-algorithm (GAMA). Here's a scenario: A user entered a search query; GAMA finds all matched keywords from all campaigns of all advertisers. Upon examining all keywords, it found that your competitors' ads take up all the first page positions, except possibly one left for you. Suppose GAMA did a calculation of expected position for each matched keyword from you and found that indeed you have many of keywords that could ranked on the first page, however, the exact matched keyword is not one of those, due to lower-than-first-page minimum bid (in fact, it is not going to be ranked in any page at all!). Now GAMA faces a dilemma: according to the published rule, exact-match has the absolute priority but the keyword couldn't show-up due to lower bid. Other keywords rank high, but picking those would violate the published rule. Following the published rule means none of your ad will be shown. If GAMA believes the high ranking of your other matched keywords indicate that you could profit from the showing of other ads, GAMA may decide to pick one from the non-exact-matched keyword. This is probably what happened behind what you saw. My point is, this is actually a dilemma for both GAMA and advertisers! It is not obvious to me what the advertisers would really benefit if Google does one-way or the other. If GAMA decide not to show any of these ad, you may never know how the search-query/keyword combination is going to perform at all. From this perspective, GAMA could be giving you a chance to learn from data so you can refine your match and bid strategies to adapt. comments?
Huayin, thanks so much for your thought-provoking question! I understand your point, but I only agree with you with respect to new keywords. The argument that until you have data on a keyword you don't know what it's worth beyond an educated guess is certainly right. Our analysts often "protect" newly launched keywords from our bidding algorithm to address exactly this phenomena. We artificially hold them on the first page for a period of time to get a more accurate sense of what that traffic is worth. However, once we've determined that traffic on that keyword is worth $0.50 per click (or whatever), it never benefits us to have Google pick out a different keyword with a higher bid to gather that traffic. Not only will you pay more for the traffic than it's worth, the traffic will end up being worth slightly less than $0.50 (or whatever) because the landing page is no longer as precisely chosen. Controlled experimentation absolutely make sense, but it's better for the advertiser to control the experiment than to allow Google free-reign. Thanks again for an excellent comment, and please let me know if you still disagree. Dialog welcomed! George
Tad Miller says:
George, I honestly think this has been happening since the beginning with Broad Match. I wrote a post in July 2007 when the Search Query Report was introduced about the same kind of canibalization. http://www.search-mojo.com/wordpress/2007/07/17/broad-matching-your-brand-on-adwords-think-again/ We have essentially decided the only way to fix it is with A LOT of exact match negatives. We have also been told directly from Google that the stronger the Quality Score the more expansive broad match will actually be, which also explains the revolving door of keywords that it keeps circulating to each "sort of" relevant keyword with a strong quality score. See: http://www.search-mojo.com/wordpress/2008/07/15/google-adwords-the-better-you-get-the-worse-it-gets/ We both work in C-ville, we really need to get together sometime and talk shop about this stuff.
Hi Tad, You're right, they've played with the dials a great deal over time since the advent of "Expanded Match" in Summer 2005. We sounded the alarm early and often, eg: http://www.rimmkaufman.com/rkgblog/2006/11/03/adwords-broad-match/ Google acknowledges (at least privately) that the "relevancy vs revenue" battle hasn't always tilted in favor of relevancy, but they maintain that they've gotten religion and have bought into the importance of serving the right ad. This bug isn't what they intended and I've been assured they're working on a fix. I'd love to grab lunch sometime to talk shop! George
Vi Wickam says:
Hi George, Do you have any update on this issue over the last year. Internal competition is something that I have wondered about, and I always assumed that Google would attempt to serve the most relavent ad, but this is definitely an example to the contrary. Thanks a bunch for your contributions. Vi
Hello Vi. I'm working on that very thing. Sadly, what was a "bug" is now a "feature."
David says:
Great post George, nice to see technology story telling alive. Kept me gripped and v interesting.


Check out what others are saying...
[...] week, the Rimm-Kaufman Group disclosed what is apparently an inadvertent bug in Google’s Broad Match algorithm. Last summer’s switch from minimum bid, to first-page minimum bid wasn’t supposed to [...]
[...] just read a very interesting blog post from the Rimm-Kaufman Group about the changes that Google had made to Broad match keywords and the resulting effects. In the [...]
[...] question, as we’ve discussed recently (Broad Match a Change for the Worse, Broad Match Controls), matchtypes are an important and complex piece of the puzzle. We believe the [...]
[...] Like Google has done before it, Yahoo is taking its definition of “advanced matching” to a whole new level! We did some digging in our search logs to see what people were searching before clicking on one of our client’s ads. We saw lots of unrelated traffic coming in on our ads.  The brand ads are riddled with far-off non-brand queries, non-brand traffic on completely unrelated searches are being served ads from our clients — a real mess. [...]
[...] careful with match types – broad match We see general terms with aggressive bids on broad being served in place of more targeted ads, [...]
[...] year about this time we identified what we thought was a bug in Google’s ad serving [...]
[...] like broad match for keyword discovery, but it certainly presents some efficiency issues if you're trying to provide ROI and hit a target CPA. SO what's the [...]

Leave A Comment