Sep 262011

Quality Score Rant + 3 More Requests

My monthly paid search column at SEL in case you missed it there:

Far too much has been written about Quality Score already and I'm loathe to add to the pile, but I have some ideas that might help us all.

The exact recipes for Quality Score on Google and Bing are unknown, and it is that uncertainty that creates so much room for mischief and blather. Uncertainty causes some smart people to rack their brains trying to figure out what's in the black box. Other folks in the scoundrel category seize the opportunity created to present themselves as the QS Master who knows the secrets that can lead to riches. At least one such pretender claimed that instead of managing bids he preferred to manage through QS manipulation. Aye Carumba!

The name itself causes problems and allows some misguided folks the opportunity to create garbage 'diagnostic' tools that identify 'opportunities' to 'optimize' one's account by identifying 'poor quality' ads.

Let's clarify a few things:

According to Google, Quality Score is Calculated from:

  • The historical clickthrough rate (CTR) of the keyword and the matched ad on the Google domain; note that CTR on non-Google sites (such as only ever impacts Quality Score on our search partners – not on Google
  • Your account history, which is measured by the CTR of all the ads and keywords in your account
  • The historical CTR of the display URLs in the ad group
  • The quality of your landing page
  • The relevance of the keyword to the ads in its ad group
  • The relevance of the keyword and the matched ad to the search query
  • Your account's performance in the geographical region where the ad will be shown
  • Other relevance factors


We can essentially combine the first three bullets with the 'relevance' and geography bullets to get: Predicted Click-Through Rate. This is the most important piece of the puzzle. When you look at the math, Google maximizes its revenue per SERP impression when it considers only bid and CTR as factors. Therefore: Any component of QS that diminishes the role of CTR in this equation loses them money, at least in the short run.

The engines aren't going to swing too far away from CTR as the ultimate measure of quality, because doing so costs them money.

Landing Page

If the landing page provides a lousy experience the user will have a bad impression of not just that site, but the experience of clicking on ads as well. Short term, this doesn't impact the engines, but in the long run it certainly could.

What landing page provides a lousy user experience? One that is slow, one that is unrelated to the search term, one that is chock full of ads.

If you run a legitimate business and the ads link to the right pages on your site, and the pages load reasonably fast, you're home free. Moving pixels around on the page isn't going to lead you to QS Nirvana.

That's all there is to it.

So if it's really all about anticipated CTR, why might my CTR be low?

It's important to create tight Adgroups and write good copy. Well worth the time to do this right.

But here's the thing: when people see a low QS they often mistakenly assume the reason is number 1 or 2, when in fact it's often 3, 4, or 5. People who don't understand this waste countless hours trying to find the magic grouping and magic wording to get them 10s. In many instances there isn't anything you can or should do to "fix" the "problem".

Qualifying Ad Copy: Let's say your business sells high-end jewelry. The mass market jewelers will tout discounts and "Ruby earrings starting at $15..." Your earrings start at $300. You'll have a higher quality score if you tout discount percentages and offers, but doing so will torpedo your conversion rate. It's likely better to qualify the traffic with "Starting at $300" to weed out the discount hunters, but doing so will hurt CTR and therefore QS. Testing will reveal which makes the most sense. As I've argued previously, this game favors the mass marketers over the high-end merchants.

User search is ambiguous:
Someone searches for "Yamaha". You sell Yamaha keyboards, but you don't sell motorcycles, or boats, or stereo equipment. In all likelihood, unless you're Yamaha corporation, your QS is going to be awful, because some fraction of people typing that query are looking for each of those and the QS for merchants of any of the single categories will have poor CTRs.

Low commercial intent: Someone searches for "Central Park". You own a hotel near central park, but your QS on that keyword stinks. In all likelihood, this is a function of the fact that most of the people typing that phrase aren't looking for hotels, trips, restaurants or anything else related to commerce. Maybe they're looking for a map, or directions, or history, but the CTR will be lousy because too few people typing that search have an interest in any specific ad.

The fact that the quality score is low doesn't mean the ad text is sub-optimal and it doesn't mean you can't profitably advertise on that keyword!

People worry far too much that having low QS keywords in the account or adgroup will hurt the QS of other keywords. The engines have a HUGE interest in preventing that from happening. Anticipating CTR accurately is a fundamentally important activity for them tied directly to their bottom lines. They're going to get this right because it matters to them, and they have a ton of really smart people working on how to get it right.

Again, poor wording and poor grouping absolutely can be problems and should be addressed, but too much time is spent applying medicine to keywords that aren't sick.


  1. The Reset Button: We've inherited accounts that have been poorly managed historically and have to fight an uphill battle initially to overcome poor QS history. Setting up a new account is somewhat of a hassle and doesn't address the piece of the problem tied to the advertiser's display url. It would be great if there was an "erase history" option that could give the new management team an opportunity to right the ship quicker. Historical data is often a good predictor of the future, but in cases where new management has take over the history can be a lousy predictor. Getting the CTR predictions right benefits the engines and the high quality managers and advertisers.The challenge is that bad actors will want to hit that button every day to start over, so I propose that each advertiser can only hit that button once a year. Maybe only the engines can hit that button based on a verified and reasonable request.
  2. End the promotional event penalty: Some advertisers have promotional events that they want to tout in the ad copy, but we find that changing copy frequently incurs a penalty because the engines have no history to go on with respect to the new Keyword-Copy combination so they erase the good history and start you back at square one. By the time the short-term promotion is over, the engines may not have figured out yet that the new copy actually boosts CTR.This is a tougher engineering nut to crack. We can't reasonably ask the engines to always assume a change is for the better because in a transition from control copy -> promotional copy -> control copy, the second transition likely hurts CTR. Perhaps copy blocks can be deemed promotional or control. The engines could then track the average CTR impact of transitions from one to the other and make better assumptions about the CTR of a new change based on that history.
  3. Give us a scatter plot: In a big program, it's tough to tell whether CTR and QS are low because of the copy (#1, 2, or 3 above), or because of the user intent. One good way to know would be to reveal a scatter plot of the QS for all the ads in the top 10 positions. Without identifying which QS went with which advertiser, we could still get a sense of whether we've done a poor job of matching keyword to copy, or whether no company scores well on this particular Keyword.For example:

    This plot would tell us that different copy could help QS. It may not be wise for us to 'fix' this copy if it qualifies traffic, but at least we'd know it's an issue.

    This plot would tell us that no advertisers have a great score on this keyword and we're likely wasting time trying to 'fix' a problem that doesn't exist.

    Some of us geeks would prefer a numerical value like standard deviation rather than graphical representation, but perhaps both would be ideal.

We all benefit from working towards better performing programs. These changes will help advertisers and engines alike, by getting better predictions of CTR and helping advertisers identify where to put their finite management resources.


10 Responses to "Quality Score Rant + 3 More Requests"
Mark says:
I love your first 2 requests that would help us all, but I fear the overall feel of your post will likely be misinterpreted by most. Immediately after reading the post I got the sense that you were saying "QS should not be the main focus of your search program" But I dont think that's what you're saying. I think what you are saying is "QS should be a major focus of any search program, though there are certain instances where a low QS can not be avoided, and advertisers can still be profitable in those instances" If my first feeling was right, I completely disagree with you, if my second was right, I completely agree with you. Some points of clarification: A)The central park example is a great example of a keyword that doesn't stand much chance of receiving a high QS. But there are 2 things here that I think are worth noting: 1)There are other types of keywords which seem overly general, but have enough specificity to make them relevant to your offering specifically. These keywords can be high volume, and will likely have a huge impact on your account, and a good search team can absolutely increase QS against these terms with consistent testing. They can't guarantee improvement, but improvement is likely if they focus their efforts on them. In this example something like "hotels new york" may be a good kw to focus on and so... 2)QS should still be a main focus for a search team, because QS improvement leads to more revenue at little to no cost, and so it's more of a KW mix approach that's important. Deciding which KWs you have control over that have a large impact on your account that could still be improved. Its important to note that QS is not just considering your CTR, its considering your CTR in comparison to the bidding competition, so if you do nothing those main KWs can fluctuate over the course of a year as competition improves/diminishes. B)Your point about qualifying clicks demonstrates the huge value of aggressive and malleable landing page testing. If a search marketing team does not have control over quickly making landing page changes, your account is officially behind the times. Improving landing pages means you can qualify clicks, and deliver the expected user experience, increasing CTR and CVR. C)You mention trying to improve QS from a 1-2 to a 10. This is a typical statement when talking about improving QS, but it makes the task seem harder than it actually is, and it misinforms people who are unfamiliar with QS. Studies have shown that improving a QS from a 4 to a 5 has the biggest impact on profit. Again, this comes back to KW mix. Dont choose to focus on KWS with a QS of 2 and expect to get them to a 10. Start with kws that have a QS of 4 and attempt to get them to a 5. I believe you would agree with everything above, just wanted to through it out there.
Pashmina says:
I second the "reset" button when a new MCC takes over an existing account. But I'm also curious as to what they mean by "historical". Is it the entire account history or last 90 days? Or some weighted combination? If you've made massive changes, and the volatility of your CTR goes up, does that affect their calculations? (I would think it should as it seems it would make them more money.) To better understand QS, I'd love an ad group (weighted) average? I want that in a "roll" up view so I know what ad group are suffering the most so I know where to focus.
Mark, thanks for your comment. I suspect we agree in principle but perhaps not so much in practice. We believe that getting the KW-Copy-Landing Page combinations right is of fundamental importance and spend a great deal of time and attention to this during the build out phase. We then test copy treatments to find the most effective "control" copy, and test whether promotional copy beats the control and under what conditions. We apply what we learn as we expand and maintain the account. In our experience, taking care to create tight ad groups at the outset with well-written copy, and testing comprehensively for the next couple of months makes a big difference. However, we find continuously fine-tuning copy is generally a fruitless exercise. Done well initially, it becomes very difficult to raise the bar through copy tweaks. That time is better spent on other types of research, KW development/ management and bidding sophistication: What is true for major e-Commerce, travel, and financial service firms may not be true universally. If you find your time productively spent making copy adjustments, great! We find it to be a waste of time after the control is well tested and the promotional routine understood, but your mileage may vary.
Pashmina, thanks for your comment! I'm not sure how they view history, and I suspect that it's done with a good degree of sophistication (weighting recent history more heavily, but not ignoring longer-term data, factoring in historic variability to assess whether performance changes are statistically significant, etc.). It's also the case that the combination of KW and copy blocks together is more heavily weighted than KW or Display URL, so new improved copy should right the ship fairly quickly for reasonably high traffic KWs, and will fix things for lower traffic terms over time. Your point about Ad Group averages is interesting, though averages can hide more granular 'truths'. More numbers never hurt, as long as they are used wisely! At the end of the day, the engines have a big incentive to predict well, so as long as we're not making it harder for them (mix-mash adgroups), they're going to do the right thing.
Michael says:
Great post, Quality Score can definitely be over-analyzed. Request #4: Allow AdWords users to see historical Quality Scores for keywords. I don't use any kind of PPC management software, so the only way for me to track change in QS over time is to manually download and save reports every day or week, which gets pretty tedious. Also, this obviously isn't a huge factor, but a friend who's been doing PPC for a long time said that he noticed QS suffer slightly when you use DKI in the ads. Have you ever seen anything like this? His explanation was that Google recognizes that using DKI often means your ad/landing page isn't perfectly targeted towards that specific keyword. I'm not sure I believe this is really a factor, but maybe it is.
Michael, good addition to the list. With respect to DKI, we've heard that said as well and test it regularly. We haven't found a definite pattern. For some advertisers static headlines work better, for some DKI works better. We advocate testing to see. Thanks for your comment! George
Ross Plotkin says:
Strongly agree with your request for the reset button on QS history. As mentioned, Google could qualify or limit this action as needed - limiting to once in set time period or only when a new MCC takes up the account or perhaps when the operator is a confirmed Adwords Exam Certified individual. This is really in Google's long term interest too as they both 1) want long term customers and 2) seek to reward relevancy: Select, new clients of mine in realizing that they are locked into average QS of 2 or 3 from their years of running very low CTR advertising do not have incentive to change under the current system. Generally, the bids needed to obtain a CTR at that juncture that would start them on the path out of the "QS pit" are unrealistically expensive. Perhaps Google has higher revenues off of such adwords customers in the short term, but there is a risk that they will simply find alternative PPC options. Instead, with the proverbial reset button they could give a fresh opportunity to such customers and increase both the likelihood that they will stay and that they will produce ads that are (more) relevant to search users.
Great point, Ross. Kind of like eliminating "debtors prisons" and creating bankruptcy protection. If folks who dig themselves a hole are locked up, proverbially or in fact, they can never get themselves out of financial trouble and can never contribute to society. It's in everyone's interest to get these folks back in the game in good standing once they've righted their AdWords ship.


Check out what others are saying...
[...] it’s true that low Quality Scores don’t mean you can’t advertise profitably on a keyword, there may be keywords that have low Quality Scores and click-through rates, don’t convert well, [...]
[...] it’s true that low Quality Scores don’t mean you can’t advertise profitably on a keyword, there may be keywords that have low Quality Scores and click-through rates, don’t convert well, [...]

Leave A Comment