Mar 152010

It's not a Bug it's a Feature

My monthly Paid Search Column at Searchengineland in case you missed it:

Last year about this time we identified what we thought was a bug in Google's ad serving algorithm.

We noticed that as we lowered bids on high traffic general terms that didn't convert well, much more specific keyword ads started being served in their place. This had three annoying consequences:

  1. The more specific KW had a higher bid, hence we end up paying more for the traffic than it's worth to us;
  2. The landing page is less targeted, so we're taking bad traffic and landing it on the wrong page, making it even less valuable traffic; and
  3. Because of the poor quality traffic pouring in on what had been a high quality term, we bid that term down meaning we also get less of the high quality traffic that the term normally draws.

Our reps at Google at the time told us that this couldn't happen, that the exact matched KW would always get precedence over the broad mis-match so what we were't happening...

We knew we were right about the phenomena and given their protestations that exact matches always won we suspected it was a mistake on Google's part. We asked them if there was logic that makes exceptions to the exact match precedence if the ad is paused. They said "yes". We then suggested that back in the day when there was a minimum bid, that an ad bid below that minimum might also be considered paused. They concurred. Then we suggested that when the minimum bid was replaced by the first page minimum bid perhaps the code wasn't updated and any ad that didn't meet that minimum would be treated as "paused". They said at the time "That shouldn't be the case; that isn't what we intended; if it's a mistake we'll fix it."

Googlers in good authority now tell me that that bug doesn't exist -- there's no reference to the first page minimum in the code that drives the rankings. Instead the explanation is simply that the more specific KW must have a higher QS than the exact matched more general KW...or that its combination of bid and QS are higher than the exact matched term. We're told that this shouldn't really happen if the QS of the generic ad is good, but our data suggests otherwise. Just a cursory look at our data showed plenty of instances where an exact match ad with a QS of 10 was passed over for a broad matched ad with a higher bid.

Managing this self-competition with the current tool set is not just cumbersome, it's impossible to do well. Adding every keyword as an exact matched negative for every other keyword in the account is unworkable. Bombing in all the general keywords as exact match negatives for all the more specific keywords is doable but time intensive and therefore costly.

Many advertisers simply give-up on broad match to prevent the shenanigans. We'd say that's throwing out the baby with the bathwater, but understand the frustration. We do what we can, but anyone claiming to have this problem solved is either delusional or deceptive.

As I noted a couple of years back, one way Google could effectively self-destruct would be to go too far down the path of allowing higher bid/lower CTR KWs to take precedence over the right KWs. As described above, not only does spraying the traffic around unpredictably make all bidding systems less efficient and hence spend less money to reach the same efficiency target, the bigger danger is in alienating the shoppers who use the ads.

Advertisers bid ads down for good reason, like when inventory is thin, and if other ads take their place and continue drawing in traffic that won't convert it is a disservice to the user as well as the advertiser.

Sometimes I rant around the office saying things like: "Advertisers should SUE! Here we've given Google instructions as to how much we're willing to pay for people who type in "Foo Bar", but when someone types in "Foo Bar" Google decides to serve my ad for "Left-handed steel foo bar" which has a much higher bid!!! That should be illegal!!!"

IMPORTANT NOTE: I should be ignored when I rant on like this -- other times, too, but particularly when I rant. Anyone looking at the Adwords terms of use for ten seconds would realize that Google can serve whatever ad it wants so there's no point in calling a lawyer.

Google's engineers genuinely believe they can algorithmically pick better ads to serve than the advertisers can. This may be true for badly managed accounts, but is not true for well-managed programs. If this notion that sometimes humans are smarter than the machines is offensive to engineers, perhaps it could be framed in the language of "crowd sourcing."

If the engineering team is willing to acknowledge that some folks might actually choose ads, landing pages and bids rationally, there may be a profit maximization angle as well. Google is not "evil", it is a publicly traded company looking to grow its top and bottom line just like us. My argument here isn't that they can't do this legally, nor is it that they shouldn't do it ethically. The argument is that this isn't a good business decision on their part.

Bing's path to victory lies not in stealing Google's organic traffic, but in taking Google's shopping traffic. That's what Cashback is about, and if Google places short term revenue maximization over long-term ad relevance they're opening the door for Bing to step through.

If average users decide that "Google is great for research, but go to Bing for shopping" Microsoft's big investment might just pay off.


21 Responses to "It's not a Bug it's a Feature"
Jim Novo says:
I've also written about this "smart client" problem, the idea that a lot of what Google "suggests" or does tends to be irrelevant to folks with strong DM skillsets. I suppose it could be true that the relative size of that skilled market is so small that Google makes more money by "overruling the ignorant". And unfortunately, it seems like that market has endless growth possibilities...
Jim, I think you're right on that last piece. To me, Google doesn't have to see this as either/or, by simply offering "Advanced" Control-your own destiny as an account option, vs the "Google knows best" tools they could have the best of both worlds.
Kenny says:
I've seen this happen too - very annoying, especially when the broad match ad that is served is specific to a particular manufacturer when the exact match query itself was generic. If the searcher is not interested in that brand, you may not get the click because you're not speaking to the variety of selection(which eventually might cause a low enough CTR to correct things, but maybe not), and even if you do get the click you're taking them to a manufacturer-specific landing page that they have to back out of to get at what they wanted in the first place. I too have been told that the exact match always trumps by Google reps, but it seems pretty clear that Google is looking to maximize click revenue at times, not relevance.
Thanks Kenny, Another particularly annoying variation on the theme involves flashing the brand ads around on general searches. The brand ads often have fairly aggressive bids, and are therefore an exciting target for "broad match gone wild".
Ken Truman says:
Right on, George. This is yet another one of the vagaries of broad matching that continues to drive smart advertisers mad. Your post prompts me to voice a peripheral complaint - Google reps often do not know how their own ad serving algorithms work! I've lost count of the number of times I've been assured by a rep that exact match kw's trump any ad rank considerations. I understand that our reps aren't software engineers, but being able to answer modest questions about what factors determine which ads are served should be a prerequisite for the job.
Ken, sadly, as Jim stated above, too few people look under the hood and raise Cain. We're very fortunate to have great reps on Google's agency team who listen and really try to get accurate answers for us. There are other account reps for some of our larger accounts who are little more than sales people; don't know how paid search works, don't know how Google's system works, don't much care, just want us to spend more of our clients' money. This may be a case where even the product engineers don't really know how the system works. With any hugely evolved, complex system it's tough for any one person to grok the whole thing, and sometimes the various pieces fit together in ways that cause unpredicted side-effects. Folks I trust and respect at Google tell me this really isn't supposed to happen. It isn't the bug I thought it was, but maybe, just maybe it is a bug, and one they might be willing to fix.
Mel66 says:
I don't think this is a bug. It's been happening for years. It *is* impossible to manage, and I can't help but wonder if that's intentional. PS, love the "foo bar" example! :)
Melissa, you're right, it's always happened to varying degrees, particularly since the advent of extended broad match. It seems to us we've gone through phases where the explanation has been: "Well, yes, we got pretty aggressive with the relevance matching for a while but now it's different." I've been jumping up and down about it since Google started denying that it can happen, which to my thinking didn't start until 2008. Prior to that, they acknowledged -- at least to us -- that that self-competition was a reality. If it's a mistake on their part I'd be delighted. Mistakes can be fixed. But I think it will take all of us who pay attention to the details to keep bombarding them with examples to prove that this is happening. Squeaky wheels get the grease...or maybe they get greased :-(
Josh says:
George - I take it you're referencing a scenario where your exact-match keywords are not listed as negative exact match keywords in your broad match ad groups?
Doesn't have to be, it can be intra-adgroup as well.
Tomas says:
I've been having the same argument with Google for months now and in the end there does seem to be a feature in the algorithm that makes it ignore matchtypes. This has been happening ever since they introduced the concept of adrank and and it's logical since bid price is a factor of adrank and you can outbid yourself. Anyway, it is only 'supposed' to happen if the resulting click on broad would be cheaper for the advertiser than a click on the exact. The people i spoke to at Google argued that this is actually doing the advertiser a favour and the tone of the conversation was quite arrogant. (the name they used for this feature internally at the time was 'broad match trumping'). But obviously the feature doesn't work as intended cause it appears as if almost no weight is giving to a keyword's matchtype when deciding which keyword triggers the impression. I was also told to just make better user of neg matches which frankly is not manageable for large advertisers. Fortunately though i do think i managed to get heard by some people at Google and it has definitely started an internal debate which has resulted in new features being considered and tested. Advertisers might just get some of the control back in the future.
If they keep hearing the same message, and seeing evidence in the data to back it up, something will have to give. There is hope on the horizon, but I can't talk about that :-)
Tomas says:
indeed, i can't talk about it either... :)
Kevin Hill says:
Is what they really need is a fourth match type. Here's google's help documentation on broad match: This is the default option. If your ad group contained the keyword 'tennis shoes,' your ad would be eligible to appear when a user's search query contained either or both words ('tennis' and 'shoes') in any order, and possibly along with other terms. Your ads could also show for singular/plural forms, synonyms, and other relevant variations. Broad match keyword: tennis shoes Ads may show on searches for: tennis shoes buy tennis shoes tennis shoe photos running shoes tennis sneakers The problem here is the "either or both words", when it really needs to be just "both words". What's the point of taking the time to bid on keywords when you'll get mapped to searches for both tennis and shoes?
Marc Adelman says:
George, You have been an advocate of "the advanced control option" for years now. Depressing right YEARS! Eh...listen up Google. From the last 2 comments about "not talking" about what's coming down the pike from Google in response to these issues, it seems that Google might be listening (half statement/half question). On one hand you probably have a huge % of the Adwords Client world running on monthly budget and not efficiency targets, pared with inaccurate tracking. On the other hand you have a few (% not count) clients and some smart agencies begging Google to stop stinting the growth and positive performance that can occur with a completly open interface into the controls of an Adwords account. Therefore Google might want to look at this as a larger client segment issue defined by client needs, rather then an exploration of bugs pertaining to issues that may or may not have occurred with the updating of the logic of ad-serving. I'm hoping that the "not talking" stand-off points to Google realizing this HUGE issue.
Kevin, Marc, thanks for your comments. Help is coming, but not the solution. There are a number of instances when the CTR on the wrong ad might be better than the right ad: Someone searches for "Sony widescreen tv", that ad doesn't have promotional copy, but the ad for 27" flat screens does. "20% off Sony Flat Screen TV" might generate a click, where the more generic, but more appropriate ad doesn't. It's still the wrong ad. And, fixing this bug isn't hard. Exact match wins, period. If the match isn't exact the CTR weighting makes sense because it's hard/impossible to define what the next closest match is, but it isn't hard to give preference to the exact match, or the highest bid exact match version if multiple versions are running. That's all we're asking for! G
George, It's all good. I think that this might be worthy of a little rant. We are all capable of it. What good is it to rant and not have a crowd to listen. Thanks a ton. Great stuff.
Alan Green says:
The best company is the one that makes the least number of errors and mistakes. Maybe Microsoft just makes less mistakes than Google.
Sam says:
Great conversation going here. Good to hear I'm not alone with this frustration. I wanted to see if anybody had tried using Google's relatively new Broad Match Modifier feature and, if so, what your experience has been. Based on Google's description it offers another level of control by requiring search queries to contain specific words in order to serve your ad. Those words are selected by putting a + sign in front of any words you require to be in the search. To use a previous example, a situation may arise in which the broad match keyword 'widget' has poor performance and is consequently bid down. The broad match keyword 'blue widget' may start triggering for the search 'widget'. With Broad Match Modifier you could theoretically fix this problem by changing 'blue widget' to '+blue +widget'. This would require any search query to contain both the word 'blue' and the word 'widget' in order to display your ad. You would still get the benefit of broad match in that a search for 'where should I buy a widget that is blue' (which would not show had 'blue widget' been changed to phrase match) but would not serve ads for the search 'widget'. It seems to be the best of both worlds, somewhere between broad and phrase match. In theory I think this could be a replacement for any traditional broad match keyword. However, we are in very early stages of testing this in the real world so my thoughts are purely theoretical at this point. Just wanted to see others had thoughts on whether the theory works out as well in practice and/or if there are situations where this doesn't completely solve the problem. Thanks in advance for your response!
Hi Sam, Yes, we’re huge fans of BMM. We’ve been Beta testing the product for a few clients and find it makes a big difference. At Google’s SEM Council last week (5/11) they unveiled the product to the 15 or 20 agencies represented, and the head of agency relations said: “There you go, George!” as I’ve been advocating for this for so long (and so loudly) I talked to the product manager in charge of BMM and mentioned that I wrote a blog post that went live that morning – they asked me to hold off until the product was officially announced – and he said: “I know, I read it!” We’re excited and expect it to roll out in the US fairly soon. George
George, I had the same fight with my reps last year when I started query mining very heavily. At first they assured me that the exact-match version should trump the broad match. They were wrong and eventually told me that if it was happening and I didn’t like it I would have to set and exact match negative. Here is Google’s official take: “In rare cases, the keyword with the highest Ad Rank might seem to be less relevant to a particular search query than other eligible keywords. Because higher relevance is generally correlated with a higher Ad Rank, this will happen infrequently. To check for it, run a Search Query Performance report. If you see an instance in which the less relevant keyword triggers an ad, add that search query as a negative keyword to that keyword's ad group.” Unfortunately this wasn’t happening infrequently. I ended up adding lots of exact match negative keywords. Solution: add the offending keyword as a negative to the phrase and broad match ad groups and move the top performing ad to the exact match adgroup (to test). I didn't change any bids. Here are the results of trying this technique on one of my campaigns. As you can see it shifted mostly from broad to phrase match. Before fix: Match Types % Total Impressions Broad 40.41% Exact 53.88% Phrase 5.71% After fix: Match Types % Total Impressions Broad 26.00% Exact 45.78% Phrase 28.23% What I learned: set your brand terms as campaign negatives in your non-brand campaigns and probably most of your head terms will have to be added as exact-match negatives in the broad and phrase match adgroups (watch your SQR).

Leave A Comment