“Everything changed,” our project lead replied, “and the documentation is pretty poor.”

In general, one of the characteristics of a good API is stability. Once published, partners in the outside world build code which depend intimately on the API spec. Well-designed APIs evolve slowly, adding and removing features gracefully so partners can adjust their apps to the improvements.

MSN jerked the rudder hard in a new direction going to v 5.1. Below is a list of pitfalls our engineers encountered. Hopefully this list will help others struggling with MSN’s docs.

Beyond MSN’s business challenges competing again Google and Yahoo, this list demonstrates some of the technical challenges MSN is facing with their codebase. To their credit, the MSN API team has responded quickly and knowledgeably to our requests. For that we are grateful. We appreciate the technical help we receive from MSN, and we hope their API stabilizes relatively quickly.

If any MSN folks are reading this, if it would be helpful, our project lead on this recent API upgrade would be glad to share his experiences with you folks on how in the future this could go more smoothly for both sides.

For contrast, about the same time period, our engineers upgraded our platform from Google API v11 to v12.

The Google API upgrade took 20 minutes.

To be fair, Google v11 to v12 was an extremely small and gradual API tweak, a handful of type changes and other little details. Comparing the Google 20 minutes to MSN’s many developer-days of effort is comparing apples to oranges. I would say, though, that Google’s API advances more gradually and more thoughtfully, making Google an easier technical partner than the other engines.

Again, our thanks to the MSN engineers for fielding so many of calls and emails to clarify the docs. The following email snippet indicates some of the places our engineers hit snags.

There there were more of these, but here is a brief list. In overall they changed the external interface, including names, enumerations and methods. This was more like a full rewrite than an upgrade. My biggest negative on the list is the documentation, as this was the cause of much of our frustrations.

And to give credit - MSN responded quickly and knowledgeably to our requests.

• .Net2/asmx (Administration, CustomerManagement, NotificationManagement interfaces) .Net3 (or 3.5?)/wcf (CampaignManagement, Reporting interfaces)
  • changed xml structure (authentication, etc)
  • enforced xml sequence
  • appears to be that the upgrade is not full, likely an internal deadline was not met
• Renamed many fields and classes to comply with industry standards
  • API is for machine communication, so the name only matters to developers, who has to redo the code now
• Changed method enumerations
  • most of these are 1-to-1 renaming enums, what is the point?
  • eg. removing underscores from EasternTime_US_Canada
• Xml arrays for NegativeKeywords (yes, no longer notkeywords)
  • fine, although web-interface uses comma separated list
• Date type of fields
  • communicate as 3 xml tags: year, month, day
  • link to docs
    • looks like a bug workaround, as field supposed to be nullable!
• Inherited classes (eg: Ad to TextAd and MobileAd)
  • readonly “Type” field is set by derived class type
  • http://msdn.microsoft.com/en-us/library/bb671952.aspx
  • creating ads as Ad xml tag with xml attribute of TextAd, but eliminate Type field during soap operations, as it is readonly…
• API DOCUMENTATION
  • bad navigation (needs to click through many pages to get to the right page, unless you know what you need, and use the sidebar then)
  • INCORRECT information
    • Campaign BudgetLimitType field for a while listed v4 enumerations! - incorrect hyperlink (simple MSN internal curl could discover these after removing the old docs from their internal website, like what we do with our website :-))
    • Parameter substitution in some Ad fields were listed without substituion (appears to be fixed by now)
  • offline version for windows framework only
  • xml code snippets are limited and started to appear in June (middle of the upgrade process)
• Why to have fields, which are reserved for future use? (like v4 had APIFlags, which never made to production, and now is eliminated)
  • Nice to see the ideas, where MSN might go
  • But has no value for an API, and makes documentation readability worse as it litters the information with useless stuff
• V5 was retired ~2 months before deadline, and 5.1 took its place (5.1 had some new reserved for future use fields, AND expanding int to long fields)
  • not a big deal, but what is the point to publish deadlines?

 

A few tidbits catching my attention:

• Facebook impressions run 13 to 16 cents CPM. For comparison, our clients in aggregate pay $10 eCPM on Google, and $6 eCPM on Yahoo. The fact FB can’t command higher CPMs speaks volumes to how advertisers value those impressions.
• Regarding the “content adjacency” problem, where ads on the social nets might run alongside content objectionable to advertisers, Chamath Palihapitiya Facebook’s VP states that the FB community self-moderates by flagging inappropriate material. Urstadt comments:
  

  Palihapitiya deliberately or otherwise, may be missing the point: advertisers dislike rude content not merely because it might reflect badly on their brands, but because the people reading such stuff are probably not thinking about buying many things that advertisers are selling.

• MySpace generated $800mil in annual ad revenue (2007 stat?) Surprising to me, I would have guessed less.
• Urstadt again, emphasis mine:
  

  Advertising on Google works because visitors come to Google looking for specific information. If a user who types in “scooter” in the site’s search field is hoping to buy a scooter, the keyword results that appear at the right of the search results can be more useful than the results themselves. In social networks, on the other hand, users show up to find friends; ads are, at best, irrelevant to that goal. The click-through rates on social networking sites bear this out. While around 2 percent of Google users actually click on a given ad (and the number is much higher when users are conducting searches for purchasing reasons), fewer than 0.04% of Facebook users do, according to a media buyer’s report obtained last year by the Silicon Valley blog Valleywag.

Worth checking out.

The article: Social Networking Is Not A Business — But It Might Be Soon, Bryant Urdstat, July/August 2008 TR Review

Let’s walk through some examples that have been valuable to our clients:

• Margin Data: For any retailer with variable product margins or who are highly promotional businesses, bidding to actual margin dollar efficiency rather than sales dollar efficiency is critically important. However, some of our clients don’t have the COG data available at the time of checkout. By providing us this data in an off-line feed, we get the benefits of margin bidding without the client having to re-engineer their website.
• Lead Valuation: For any business using its website to collect leads it’s important to incorporate back-end data on the ultimate value of those leads into the bidding and analysis. For banks and lending institutions that lead quality may be measured by account creation and funding, credit scores of applicants, or other well established indicators. For businesses with long sales cycles the information may be more qualitative and can be as simple as a “Good” or “Bad” flag based on the initial call with the sales rep. Any information that can meaningfully distinguish between the more promising leads and the less promising ones will be valuable to a good bidding system.
• Customer Type: Some companies cater more to businesses than consumers. They know that the lifetime value of a business customer is much greater than that of an individual. Knowing that certain types of keywords drive more business customers than others is therefore an important consideration in setting bids and evaluating success metrics.
• Credit Allocation: As with margin data, some clients pass us information in the pixel about which marketing program their systems think drove the order. Others pass us that information after the fact. We prefer to “see” all the orders regardless of the client’s order allocation so that we can help add insight into those difficult allocation issues. Our clients have benefited from discussions to the effect: “We noticed that X% of orders your system is allocating to affiliates came through non-brand search ads within 30 minutes of the order — does that make sense?” We can then credit or discredit those orders as the client sees fit.
• Match-Back Data: For good reason, many catalogers and other direct marketers are more interested in attracting sales from new customers than existing customers. If this is important to your firm, feeding that information into the bidding process, so that more value can be placed on ads that tend to generate more new-to-file customers, can be a big win.
• Frauds and Cancels: Many retailers, particularly in electronics and jewelry, see relatively high fraud and cancel rates. Using a back feed to knock out credit for orders that didn’t stick can be important, particularly if some types of keywords are more susceptible to these problems than others.
• Anything Else: There aren’t many limits here. Whatever information relating to the above, some combination thereof, or anything else that differentiates value can be handled with back feeds.

A top-tier bidding system sets bids based on the observed value of the traffic generated by each ad, phrase, or collection of related phrases. For many advertisers, the true value of the traffic isn’t perfectly clear at the time of the order. The ability to take information after the fact and fold it into the bidding algorithm can be the difference between success and futility in PPC advertising for these firms.

Previous posts in this series are here:

Bid System Imperative #1: Strong Statistical Foundation
Bid System Imperative #2: Deep Flexible Tracking

Here are the year-to-date ad spend share figures for our clients.

Same data, graphed:

Background: RKG manages pay-per-click advertising for over 100 online retailers. Most are B2C. Nearly all bring a direct-marketing perspective to paid search. Our clients focus on ROI rather than CPC or CPM. We’d suggest online retailers as a class are among the savviest and most demanding competitors in the click auctions. Data from our agency may or not be representative of the total online advertising universe.

Our clients don’t establish a priori budgets by engine. Rather, our clients instruct us to buy as many clicks as possible which meet their marketing targets (revenue, margin, ROI, etc). Our portfolio bidding platform is highly effective at achieving these targets.

Thus, we’d suggest that the ad spend share data presented above is a reasonable proxy for click quality. Click quality is a function of both the population using each engine, as well as the engines’ skill at matching searches with relevant ads.

The main take-away from those data is Google’s staggering dominance of paid search. Sometimes this obvious point seems forgotten in the mainstream business press. In the paid search world, Google isn’t big, Google is gigantic. And Microsoft isn’t small — Microsoft is tiny.

It seems logarithmic, like the Richter scale. In terms of share of ad dollars, Yahoo almost is four times larger than Microsoft, and Google is almost five times larger Yahoo. Rarely is the skew between the top three competitors in any field so dramatic.

A smaller take-away from these data is Yahoo has upticked in each of the last four months, gaining share at Google’s expense. However, the absolute size of that improvement is small, and doesn’t change the situation.

Which engines are driving sales to our clients? Because our analysts and our technology manage campaigns to profit targets, and because those targets are typically consistent across the engines, it follows that our clients see roughly the same percentage of revenue from each engine as the corresponding ad spend.

Here’s that graph, tracked web sales by engine by month, aggregated across all our clients.

Are the engines equally good at transforming impressions or clicks into ad revenue? Nope.

Google brings in an effective aggregate CPM of $10, versus Yahoo’s effective CPM of $6. Microsoft commands an impressive $20 eCPM, but remember that is on tiny volume.

Google enjoys an effective aggregate CPC of 52c across all our clients’ campaigns. Yahoo and Microsoft come in about 17% lower, at 44c and 43c respectively.

Back in April, Yahoo began a test with Google running Google ads on something like 3% of Yahoo impressions. This was undoubtedly a “proof of concept” test, and apparently both parties were pleased with the results. Why wouldn’t Google be pleased? They picked up additional revenue simply by adding another, more substantial than average, syndication partner for its “AdSense for Search” product. Yahoo clearly made more money serving Google ads than it made serving its own.

If federal regulators let this happen – a pretty big “if” – Google and Yahoo both win, and Microsoft would lose out. Yahoo will have successfully fled from a hostile takeover bid by Microsoft by throwing itself into Google’s arms.

But what about the advertisers?

Danny Sullivan asked me to opine on the impact of this, and the article is over on Search Engine Land.

Let me know what you think!

We take our aggregate spend on Google, Yahoo, and MSN across all RKG clients and show how it breaks down by month, by engine. (Search and Content combined, but the data is heavy on Search) In addition, since advertising spend is set on economic goals and not engine favored, this approach removes search engine bias, seasonality and RKG growth. In other words, the allocation of spend reflects availability of quality clicks on each engine, and we’re using relative proportions.

May 2008 results: steady, no big change.

Google @ 80.8%
Yahoo @ 14.5%
MSN @ 4.7%

Google’s at ~81% because their clicks work. As for market share, no big change YTD.

Here’s a glimpse at CPCs across our client base for the month.

The results: Google earns more per click.

Next, CPC trend by month, indexed so google = 100%. (This removes seasonality and RKG growth.)

Pretty stable. Google consistently has 10 to 20% higher CPC than Yahoo, MSN.

CPM (cost per thouand imps) for May across our clients:

MSN actually earns more per imp than Google, though on far far far fewer imps and clicks (our clients in aggregate bought 14 times as many clicks on Google vs MSN in May).

Trend in eCPM, indexed so G = 100% to remove seasonality and RKG growth:

Reveals that as we buy more traffic from MSN, we see eCPM decreasing.

Stay tuned for next month’s post!

Now Microsoft has launched the next salvo in its attack, cutting a deal with HP to make Live the default browser on new HP machines.

From MSFT press release yesterday:

“Microsoft Live Search Toolbar to Be Distributed on 2009 HP Consumer PCs”

REDMOND, Wash. — June 2, 2008 — Microsoft Corp. today announced that it has won a key distribution deal with HP, the world’s largest PC manufacturer, to install a Live Search-enabled toolbar on all HP consumer PCs planned to ship in the United States and Canada, beginning in January 2009. As part of this deal, the default search engine setting in the browser on all HP consumer PCs will also be set to Microsoft Live Search.

Same day, different press release. From ComputerWorld:

Firefox on track to crack 20% share in July

June 2, 2008 (Computerworld) Mozilla Corp.’s Firefox browser is on pace to hit the 20% market-share mark next month, a Web metrics company said today. Firefox boosted its share by 0.6% in May, accounting for 18.4% of the browsers used during the month and putting it within shouting distance of a major milestone, according to Net Applications Inc. “Firefox is trending to hit 20% market share sometime in July,” said Vince Vizzaccaro, the company’s executive vice president of marketing, in an e-mail.

Interesting. Right off the bat, 1 in 5 of Microsoft’s pricey HP Live users are back to Google.

Dunno. Still think this game all shakes down to search quality.

Or, to be more precise, to perceived search quality.

Which Mountain View has locked up.

If you’ve ever wondered how Google decides questions ranging from how many results to serve on a page, to the amount of whitespace with which to pad the logo and the most effective color with which to tint the sponsored link box at the top of the page, the answer, not surprisingly, is A/B split testing.

Among the points Barry found most interesting are these 3:

* The more search results per page, the less people search
The reason? Less search results mean faster returned results.
* 30 results per page would result in 20% less searches when compared to 10 results per page.
* Google cut the bloat out of Google Maps and noticed a 30% increase in usage

The implication is clear: When it comes to Website Effectiveness, your site’s speed matters, a lot.

You’ll find more nuggets Mayer shared about Google’s design process in the News.com article: We’re all guinea pigs in Google’s search experiment.

APT provides you more information about your search campaigns than you’d have otherwise. But some advertisers mistakenly believe APT reveals more than it actually does.

I’d argue the problem doesn’t rest with Google, but rather with advertisers’ mental models of search results pages (SERP s).

OK. APT lets you see a “clean” search results page for a given keyword, Google domain, display language, country, and geography. For example, say you are a US Google advertiser based in NYC. APT lets you check out your local ads in different regions nationwide without leaving your desk.

Some paid search advertisers (or their managers) search on their own company’s terms regularly (sometimes obsessively) to check the position of their ad relative to their competition. Google does report daily average position data, but that dry stat doesn’t reveal competitors’ ranks, and doesn’t provide the visceral satisfaction of “making sure myself that our ads are where they should be” by pulling up the “actual” SERP.

But manually searching your own terms from one computer to check your own ads may not tell you the full story.

I mentioned the geography issue above — sitting in NYC, you shouldn’t (and wont) see your San Francisco campaigns.

Also, Google adjusts ad serving based on IP click-through rates. If a given IP or (perhaps even user) shows a statistically significant anomalously high or low CTR for a given ad, Google may stop serving it.

An example: a major retailer with a strong focus in the (sorry for the obfu) widget category spends large sums advertising the single word term “widget” on Google. As a result, their managers often search for “widget” to check their ad and their competitors’ ads. Obviously, they seldom click their own ad. Because the CTR on this high-impression ad is effectively zero from within their corporate IP range, Google stopped showing their ad to searchers at the corporate HQ building. This caused some valid concern (”Why aren’t we showing up?!?”) until explained.

Enter the Ad Preview Tool.

“Hallelujah !”, advertisers rejoiced, “a way for us to get to the ‘true’ SERPs, cleansed of geo effects and individual history effects and all other Google muck. Like the platonic simplicity of GoTo’s CPC-displayed-on-SERP approach. Now we can keep an eye on on ads’ true rank (and adjust bids to hang in position three). Hallelujah!”

Beyond the parenthetical bid-to-position stupidity (B2P nearly always trashes advertiser profits ), the problem with the prior paragraph is the notion of a true or pure or absolute SERP.

Because in 2008, my SERPs often aren’t your SERPs.

Google knows too much about me. Where I live. What feeds I read. What I’ve searched on before. What certain terms mean to me. Fast: what is a bass? For you, fish or instrument?

Google also varies SERPs to test new ads. Google burns a small fraction of impressions “over-serving” baby ads so as to determine their QS to price ads “correctly”. I’d wager this sampling is essentially random over time, users, and data centers. Again, this leads to some differences in SERPs between users.

Which brings me to my main point:

There is no such thing as a platonic or “true” universal SERP for a given phrase.

SERPs vary to some degree by user, by time of day, day of week, by geography, by IP, by Google data center, by user search history. Who knows, maybe by Google market cap and phase of the moon. What is certain is that there are many many factors involved.

Sure, if you set an outrageously high maxbid, say $25 or $50 for “widget” (this time not obfu, just generic) and don’t set a campaign budget cap and do have a well-known brand name (thus high CTR) and do have OK ad copy and do have fair prices on your merchandise and and decent selection and do have reasonably usable site, then yes, with very high probability you’ll effectively “own” position #1. And that would hold for just every user and every SERP in your geography. (Dell takes this approach for many computer terms.)

Dominating the top may or may not make economic sense, may or may not make branding sense, but you can get there if you want. Just spend enough. OK. In the case of one advertiser determined to outspend all others, yes, the “true” SERP will always have big-spending-advertiser in the first position.

But that atypical. Usually several savvy big-spending advertisers jointly share the top of the page, jostling up and down a slot or two based on bidding algorithms, management ego, whatever. As positions vary by bidding and are filtered through user geography and user search history, there’s no “true” PPC SERP in this case. And I’ll argue that that is OK.

“But wait,” you may be saying, “doesn’t APT reveal the True SERP?”

I think not, as APT also filters on the user data (here “user” means APT user, the advertiser). Let’s watch this in action.

I turned off cookies (FFox: Tools >> Options >> Privacy >> Uncheck “Accept Cookies From Site” box), logged out of Google, deleted existing cookies (FFox: Tools >> Clear Private Data >> Check “Cookies” box >> Clear Private Data Now”) , closed all browser windows, and then opened a new browser. Clean as a whistle. I then went to https://adwords.google.com/select/AdTargetingPreviewTool directly.

In the tool, I specified “plumber” as the keyword, searching on “google.com”, language “English”, country “United States”, “All regions within this country”.

One would expect APT to return national search results for “plumber”. After all, that’s what I asked for, and there are no cookies involved. Here’s the result page.

Despite my request, the page is clearly geo-targeted for my town, beautiful Charlottesville VA. Half the paid ads are local ads: position 2, 4, 5, 7, and 8. So Google uses IP data, which we’d expect.

(Aside: I feel sorry for the guy in position 10 who missed the concept of local targeting. Errr, C’ville is 750 miles from Grand Rapids, friend. Here’s a sympathy backlink:Grand Rapids plumber )

How about trying APT with hiding our true IP? Turn off and purge cookies, close and restart browser, activate Tor , and back to APT

Hello from Austria! That’s where I came out of the onion, it seems. OK. Complete the form (”plumber”, “United States”, “google.com”, etc). and submit:

APT doesn’t like anonymous proxies. Banned! Couldn’t use APT for about a day. (Unless I was logged in with cookies on, then it would let me through.)

So let’s wrap this up. My suggestions:

• Don’t get hung up trying to determine the universal “true” SERP for a given search phrase. It doesn’t exist.
• Realize my SERP might differ a bit from your SERP for the same phrase at the same time.
• Realize even your SERP for the same same phrase might vary a bit when you re-query in a short interval.
• If you want to know where your ads are on the page, rely on Google’s average position stats. Averages are powerful summaries of the whole population of observations. Use them.
• If a term is performing above your economic hurdle and it isn’t already at the top of the page, spend more and move that ad up! (And then ask yourself why your bidding system allowed it to get out of position…)
• On the other hand, if a term is performing below your economic hurdle, lower your bid! (And again ask why your bid system overspent on it…)
• Do use APT to monitor your local campaigns in different geographies.
• Don’t use APT to monitor your national campaigns.
• To avoid getting banned, avoid the sneaky stuff (using automated scripts to scrape APT, using anonymizing proxies, etc)

Happy Monday, all!

Bonus tip: our Google reps confirmed that the natural results on APT are “authentic”, too. A small SEO tidbit there…

From Techcrunch back in October:
Google assigns dollar value to search results

From SEL back in April:
Google Showing Ranking Scores On AdWords?

From Search Engine Journal, same event:
Google Leaks Quality Score Variables (Pscore, mCPC and thresh) in Search Results

Hattip: SEOBook