Jun 152009

Quality Score: the Hype and the Reality

This Post has been Updated to correct some misinformation I was inadvertently spreading!

The volume of misinformation and hoopla surrounding Quality Score (QS) is astounding!

Some of the kooky ideas I've heard and read recently:

  1. Quality Score is a measure of how well the landing page relates to the user search.
  2. Utterly False: Web design shops may be guilty of propagating this myth, but it's certainly persistent! Landing page plays the following role: if your landing page is just a collection of links to go elsewhere (usually sponsored links) you're an AdSense Spammer and your QS will drop to zero. Good riddance! If your landing page loads really slowly, that will hurt your quality score. That's it, nothing else. As long as you're a credible destination and not crazy slow, landing page plays no role whatsoever.

  3. Quality Score is a black box with lots of mysterious elements.

  4. Not Really: The exact mechanism for calculating QS is proprietary, of course, but the elements in play are not: measured/historic CTR of the Keyword-copy combination , measured/historic adgroup CTR, measured/historic account CTR, measured/historic domain CTR. From these elements Google calculates a best guess for the CTR of the ad in question. This is normed for position, calculated only from google.com traffic that matches the KW exactly and fixed on a scale of 1 to 10. More on this final bit later.

  5. The advent of Quality Score represented a major change for Google
  6. False: The notion that positions should be determined by Bid x CTR was the brilliant insight that maximized Google's revenue per impression per ad. Any step away from CTR would reduce their revenue. Quality Score had a major impact on AdSense Spammers and Affiliates, but changed very little for legitimate advertisers. For legitimate companies this represented a shift from measured CTR to predicted CTR (one and the same given adequate volume).

  7. The way CPCs are calculated has changed dramatically
  8. False: The magic formula revealed by Google's chief economist is exactly the same as it was before with the respective QS in place of the CTR. As long as we understand that QS is a linear function of CTR, so that a QS of 8 means you have twice the predicted CTR of a QS of 4, there is no difference in the calculation pre- and post- QS launch.

  9. PPC Programs can be managed simply by focusing on QS and paying little attention to bids
  10. Are you serious?!? We actually heard someone make this claim on stage at a conference. Writing compelling ad copy is important. Highlighting differentiating reasons to shop with you is valuable. However, once the copy is tightly targeted, and the messages tested and compelling, there are no more levers to pull. It's very difficult to beat well-written control copy in a test. Bids can be changed dynamically and smartly to respond to changes in traffic value, there is only so much tuning you can do with copy. If you don't have a flexible, powerful bid management system perhaps you don't have any other levers to play with, but if you do have a system you'll get far more bang for your buck analyzing, reacting to and anticipating changes in performance data through proper bid management.

For legitimate advertisers, QS is simply a normed system for evaluating an ad's CTR. One of the really clever features of this normalization is that the advertiser can get an immediate sense of whether the CTR is good, bad or indifferent and that's a huge boon.

In the past if you saw that a KW-copy combination has a CTR of 3% and it's in position 3 on average, there's really no way to know if that's good or bad. Not only did the position impact the raw CTR, making it hard to know whether the copy or the position was driving the number, the range of "good" CTR changes dramatically based on the KW. The tighter the KW defines user intent the higher the range of CTR will be for the sponsored links. The less clear the lower the range. Someone searching for "Yamaha" could be researching motorcycles, pianos or stereo equipment. Someone searching for a "Yamaha FZ6R" is pretty clearly looking for a street bike, so the match between advertiser and user will be very tight.

Normalizing the CTR so that you know at a glance ads that have a good CTR (QS >=7) or bad CTR (QS < 5). This lets you identify the KW with problematic CTR performance easily. There may not be an easy fix for low QS. If your competitors pitch "Low Price Guarantee and Next Day Shipping" and you can't match those offers, then you may not be able to win the copy battle. But, sometimes tightening the copy for a particular cluster of ads can materially impact performance. Normalization has made management easier.

Quality Score is important, just as CTR has always been, now we just have a bit better ability to identify and address the problem children.


10 Responses to "Quality Score: the Hype and the Reality"
Jennifer says:
In response to #5: Google has no QS data tied to Campaigns or Accounts This is not what the help section of Google Adwords reports when talking about the components of quality score. Here is what Google's help section states about the affect that account level history has on QS: Your account history, which is measured by the CTR of all the ads and keywords in your account http://adwords.google.com/support/aw/bin/answer.py?hl=en&answer=10215 Is your point that this isn't as important as the other components? Account history certainly does have at least some influence.
semmy says:
Great post George, I like the way you explained everything. I have two questions though...I hope you can give me your opinion and clarify it for me... 1) You said about QS as Black Box: "...This is normed for position, calculated only from google.com traffic that matches the KW exactly and fixed on a scale of 1 to 10. More on this final bit later." Scenario: lets say that we want to target Mexico and USA, both countries with english KW's and ads, as we don't know spanish... Does this mean that there is no need to split campaigns by countries and hope for better QS in USA, as QS for Mexico is also calculated from google.com traffic? 2) I heard few AdWords representatives saying that if you had really bad QS on KW's in campaigns, that sometimes it is better to close that account and start from scratch, as that account has history that can influence future KW's and their QS." I don't know what to think now?
Jennifer, Semmy, thanks for your comments. We've grilled the folks at Google on this point all the way up the food chain, (#5) and that is the word we've gotten. Setting up a new account wipes out the AdGroup level QS, but the KW-Copy combo, and the domain QS carry-over. I'm told that the folks at Google read our blog, but I'll shoot them an email to confirm that I'm not propagating yet another myth. I'll let you know if I'm wrong. Semmy, we haven't run ads in Mexico, so I can't say for sure. The country distinction is an interesting question. I'll ask. George
Marc Adelman says:
George, Great post as always. Quality score has found its way into what I call a "Wizard of Oz" conversation. You know, a "We don't know what's behind the curtain, but it sure is magical" kind of conversation. The black box holds ultimate power because it can never be defined kind of conversation. It drives me nuts, and from this post, I think it drives you nuts too. This all started back in Q3 2008 when Google did make a update to the QS system. http://adwords.blogspot.com/2008/08/quality-score improvements.html In this release they outlined 3 major changes in the quality score system. >Quality Score will now be more accurate because it will be calculated at the time of each search query >Keywords will no longer be marked 'inactive for search' >'First page bid' will replace 'minimum bid' in your account The overall concept of Quality Score did not change. The Adwords world was not reborn. Just improved. I think there has been a lot of hype about these changes as if they "Changed the Game". They didn't change the game, but they did allow the Adwords marketer to jump out of the world of Poor - Good, and into a slightly more granular understanding (1-10) of Quality Score - Still a bit Fred Flinstone, but not total Neanderthal. I hate to bring a car analogy in here but I can't help myself. It's easier to fine tune a race car if you can measure in actual MPH rather than - slow, kinda fast, and blazing fast. Here to, it opened a window for quality score to enter into optimization logic more then before. I totally agree that there is a lot of myth and hype about QS and the best thing we can do is drag the facts, as you did into light. All of this being said, I agree with most of the things you wrote.... except: 1. Google Accounts do have a Quality Score. This is a black box QS and factors in several historic metrics. We stumbled our way to this hidden truth through some issues we had with Google. We didn't accept the general understanding of this - or what we were originally told because we had some gaudy numbers staring us in the face that told us otherwise. Once the Google programers of this black box Account level quality fessed up what was behind the curtain, it justified the data we saw on our side. The same Campaign -Adgroup - KW - CPC - Adtext relationship was carbon copied from an average QS Account to a high QS Account and our world changed. CTR increased slightly, but what jumped off the page was a significant increase in impression share. Through several tests and tribulations, we firmly came to understand the relationship between quality score and impression share. That meant that not only can a high QS lower my avg cpc, but it can simultaniosly increase the amount of people that see my ad in the first place. Additionally high QS KWs in aggregate, contributed to a raising of the Account QS. This meant that one high QS KW can positively have an indirect relationship on another KW in the same account. That was game changing for us. 2. The structure of your KW to Adtext relationship can drastically effect QS. Account and Campaign structure has no effect - I agree. Through several tests on adgroup structure - specifically the relationship between each individual KW and the Ad copy, we discovered that one could effect QS through creating the most granular and relevant relationship between 1 individual KW and its corresponding ad copy. Therefore we took KWs that we already had - tweaked the structure as stated above, and saw an increase in QS which of course led to a decrease in cost, increase in position and CTR, and as stated before an increase in impression share. CTR does have a linear relationship to QS. I agree. But there are ways to increase CTR beyond conventional methods, that can have an indirect, yet powerful relationship to increasing QS.
Marc, I stand corrected. I'm editing the post to correct the misinformation. Rats, and I'm trying so hard to clear up the mysteries... G
Tom Hale says:
So you are of the opinion that the relationship between things like KW, Headline, Ad Text; and how they relate to destination URL and landing page elements has little or nothing to do with overall QS? Seems to me basic SEO gives you a edge in relevance. If things like URL, Title Tag, Headers, of a landing page are in sync with the KW and ad elements seems to me that impacts QS. Or should. You are saying that the link to content ratio, and page load time, are the ONLY landing page elements that effect overall QS. I have my doubts, based mostly on observation. If Google doesn't currently figure the overall relevance of the KW-Ad-Landing Page matrix into QS, bet it is on the drawing board. I hope this isn't true - That you can tie an ad about cat food to a KW about cat food. And that you are just as likely to get a QS of 10 if you land on a page about used cars as you will landing on a page about cat food. But that is what I am hearing you say, as long as the used car page isn't deemed "link spam", and as long as the page loads promptly enough there is no difference in overall QS between the "cat food-cat food-cat food" matrix as there is the "cat food-cat food-used car" matrix. I Google really oblivious to post-click relevance? -T
Hi Tom, Google doesn't need to care about post-click relevance, that's our jobs! Obviously landing the cat food ad on the used car page is going to generate lousy Sales dollars per click, which will force bids down and drop the ad off the page. Folks not paying attention to bidding and revenue per click will lose their shirts, but the net effect for Google is the same. SEO tactics carry their own benefits, but QS isn't one of them. Choosing pages wisely, and testing benefits conversions and allows more aggressive bidding. Happy hunting!
Yesi Hill says:
hi, great post. You have explained the things so well. good job. Thanks for posting.


Check out what others are saying...
[...] Quality Score: the Hype and the Reality — 1:44pm via [...]
[...] treści artykułu możemy znaleźć na Quality Score: the Hype and the Reality 6. Wynik Jakości nie ucierpi jeśli Twoje reklamy będą wstrzymane, bądź będą wyświetlane [...]

Leave A Comment