How can you support this claim?  Can you discredit the possibility that the apparent improvement is just noise? How can you apply that authoritative label of “significant”?

Here I’d like to walk you through a basic test of significance that you can use to de-mystify changes in your paid search data.

If you’d like to skip the math, click here.

Let’s start with a situational example… say you’ve added Google Site Links to your brand ads and you want to show that brand click-through rate (CTR) has improved as a result.

1. First, you need to know what value brand CTR is potentially improving from.  Let’s call this value mu (pronounced myoo), and you can choose it in a variety of ways: the average or median CTR over the past month, the average or median CTR from this time of year last year, etc. It should really be whatever value you believe CTR to truly center around.
2. Next, you need data points.That is, you need several days of CTR data since the Site Links have been running. How many days is up to you. Generally, more is better, but I’ll touch on that later. The number of days you have is n. Take the average of the CTRs from those days; this is called xbar. Lastly, take the standard deviation (excel function stdev) of these CTRs and call it s.
3. Now we can compute a t-score, and with it, the probability that the change in CTR you’re seeing is or isn’t attributable to chance. Set t = |xbar – mu| / (s/squareroot(n)). Then use the function tdist in excel, and for the arguments, plug in t, n-1, and 1. The number that this function returns is the probability that the change in CTR is simply due to chance, aka noise. If this probability is very small, then we say CTR has changed significantly.

Enough Math! Is The Change In My Data Significant?

I’ve prepared an excel spreadsheet that handles the arithmetic. In this model, change the gray shaded cells to reflect your data. Enter the data that you think has fundamentally changed in column C. Only include data points since the change began. Then, in cell G2, enter the value from which you believe the data to have changed. That is, the average value of the data before the change.

The value p, produced in cell G7, is the probability that the change you’re seeing is only due to chance, and thus meaningless. Typically, a p-level must be below 5% to be considered significant. (If you want to be super, super sure, you can use 1% or 0.1% instead.) In other words, if your p-value is 5% or less, you can confidently say that the change in your data is real, definite, and due to something other than statistical noise. It’s a pretty safe bet that whatever initiative you took – whether it was switching landing pages, altering ad copy, or refining your bidding – was the catalyst for the improvement instead.

Allow me to fill in the spreadsheet with an example. For an imaginary online retailer, brand CTR hovers around 4.4%, so I fill in cell G2 with the value 4.4. The retailer enables Google Site Links, and CTRs for the 3 days afterward are 4.3, 5.2, and 5. So I enter those three data points into column C. And voila… the p-level comes back as 12.66%. This says that there is a 12.66% chance that the rise in CTR was due only to noise.

Not significant. Sorry, click-through-rates haven’t really increased, or at least, we can’t be very confident that the observed change is anything more than random noise.

But… three days is not much data. As smart analysts, we are cautious when examining trends over only a few days, and this significance test incorporates such wisdom. As the number of data points (n) you use increases, p-levels fall. For example, if all the numbers in the above example were the same except that you used 7 days instead of 3 (so n=7), the corresponding probability drops to 2.6%. In this instance, it’s very unlikely (2.6% unlikely) that the increase in CTR was due to noise, so here you can rather confidently say, “Yes, CTR has increased, and it wasn’t due to chance. It was probably due to the site links.”

Technorati Tags: Paid Search analytics, Paid-Search-statistics

According to one industry observer, comScore is about to release numbers indicating Bing’s share has jumped 0.6% in the last month and a half, while Google and Yahoo each lost 0.3%. Now share of search volume and share of advertising revenue (factoring in syndication partners) are different animals, but from our data, comScore may be under-reporting the shift.

These data are aggregated so our higher spend clients dominate the results. We’ll take this apart to study medians later.

MSN/Bing’s growth from 5.3% in February of 2009, to 9.0% in February to date is highly impressive. Where much of that growth seemed to come at the expense of Yahoo early on after Bing’s launch, January and February show Google starting to feel the heat. Google’s share of paid search advertising dollars dropped from almost 81% in December to under 78% in the first week + of February.

Questions abound:

• Did Google buy a Superbowl ad because they’re seeing erosion of paid search share?
• Is Bing’s growth solely a function of Cashback, and is Cashback financially sustainable for them?
• While we and others have criticized Bing’s commercials, are they starting to have an impact?
• Has Google been distracted by all its other endeavors and taken its eye off its core product?
• Is there something screwy in our data that we haven’t accounted for?

Given that I pulled the data myself the odds of the last question being salient aren’t bad. One of our far more careful, thoughtful, judicious analysts will probably “tut, tut” me later.

Folks who budget search, and go so far as to budget by engine, might cause a shift in share simply by executive decision. But that’s not what we see in our data. Our clients don’t budget for the most part and don’t budget by engine at all. We’re spending more on Bing because the quantity and value of the traffic dictates spending more.

Paid search is only one piece of the advertising mix for Google, but it is a big piece. If this trend continues expect to see some refocusing from Google.

Love to hear if others are seeing similar trends!

Technorati Tags: Bing Market share, search engine market share

• Bing Cashback Basics Bing Cashback pays users to make purchases using Bing. Here's how it works....
• Bing: A Closer Look at the Numbers A more careful study yields a different perspective....
• Bing Cashback: The Ad Campaign they Should Run Microsoft is willing to buy share and Google may give them the opportunity....

The big number that jumps out is a 15% Y/Y decline in search advertising revenue.  From Yahoo:

Marketing services revenues from Owned and Operated sites were $971 million for the fourth quarter of 2009, a 9 percent decrease compared to $1,063 million for the same period of 2008.  The decrease was primarily driven by a 15 percent decline in search advertising revenue and a 1 percent decline in display advertising revenue.

Results from a basket of RKG clients are in line with that figure with a spend decline of 20% across Yahoo and its partners in Q4.  By comparison, RKG’s spending on Google rose 14% Y/Y.

While a general improvement in the economic environment helped all three engines’ Q4 numbers, Yahoo continued to bleed share to Google and Bing at a fairly steady rate according to our data:

Interestingly, Yahoo’s reported numbers were bolstered by a relatively impressive 6% Y/Y lift in revenue from affiliate sites.  While they don’t break out specifics, it stands to reason that a big portion of that revenue is from Yahoo! Partners on search.  It’s unclear if the 15% decline in search advertising revenue includes the partner results; If so, it’s an even more impressive gain, but one with not so obvious costs.

To Yahoo’s credit, they have offered advertisers the ability to block poorly performing partner sites for some time now; however, if most advertisers are not parsing their data this finely, the expansion of Yahoo! Partners with lower and lower quality traffic will continue to drive down ROI and ultimately spend on Yahoo O&O sites where the bulk of their revenue potential is.  Advertisers and agencies taking advantage of Yahoo’s partner exclusions (like RKG) benefit from this scenario through reduced competition in bid auctions and lower CPCs.  Good for our clients, bad for Yahoo.

Fortunately for Yahoo, this effect is likely to diminish over time as more advertisers wise up to just how much money they are throwing away to poorly performing partner traffic.  Yahoo has also made a smart move recently by improving advertisers’ ability to manage bids on their network.  In addition to the existing all-or-nothing exclusions, we can now apply a percentage bid adjustment to partner traffic.  This is a very welcome move and the benefits should flow both ways in the long run.

Technorati Tags: financial, Google, paid-search, partners, PPC, share, syndication, Yahoo

• Paid Search Market Share by Engine: Yahoo Resurgent? We took a look at "Market Share" of the big three engines over time and saw some interesting trends....
• Yah Hoo for Yahoo! Yahoo's new syndication bidding controls should be a big win for everyone!...
• eBay Adopting Syndicated Google Paid Search Ads, Dropping Yahoo Has eBay been phasing out syndicated Yahoo ads in favor of Google's? RKG records indicate a major change in the...

Well, a lot has happened in the mobile space in the past year:  Apple sold another 20 million or so iPhones worldwide, Motorola and Verizon launched a $100 million marketing campaign for the Droid, AT&T claimed a nearly 7000% increase in data use over the last few years and Google finally debuted its self-branded Nexus One.  It’s a good time to revisit our numbers.

As we did previously, we are looking at click traffic from mobile devices with full internet browsers that trigger standard PPC ads.  In December of ‘08 we found just 0.4% of PPC clicks came from mobile users.  By October of ‘09 that figure had already jumped to nearly 1.3%.  (Although it’s dwarfed by Google, it was around this time that Yahoo began showing its full search ads on iPhone-like devices.)

In early January of this year, mobile’s share of PPC clicks is approaching 1.8%:

Traffic from devices running the Android OS has increased the most dramatically in recent months, with Android’s share of mobile PPC clicks rising from 6.3% in October to 10.5% this January.   Android’s quick rise corresponds with Motorola’s Droid launch as well as the launches of a slew of less-heralded Android devices late last year.

Share of Mobile PPC Traffic for Select Devices:

At the same time, iPhone’s mobile paid search traffic share (including iPod Touch) has been dominant, but somewhat flat, while Blackberry is down significantly.  It will be interesting to see if the Google Nexus One launch gives Android another big bump despite the reports of anemic first week sales.

So, we have this big increase in mobile traffic, it must be a boon to advertisers, right?  Unfortunately, it doesn’t look that way.  Conversion rates are very poor compared to traffic from desktop/laptop computers.   There are a couple of ways to think about the low conversion rates:

• Conversion Rates are Better than they Appear
  1. Cookie breakage as people browse on their mobile devices, but purchase on their home or work computers. 
  2. On some devices, there’s also more likely to be technical issues with tracking cookies or scripts that we don’t face on full PCs or Macs.
  3. People click on the ad, find the store nearest them and purchase in person.
• Conversion Rates Really are Awful
  1. Shopping on a tiny screen with two-thumb typing may be too much of a barrier for folks to overcome.
  2. People click on your ad and walk into someone else’s store to make the purchase.
  3. People are in someone else’s store want to compare prices online and click on your ad to show the clerk and get the benefit of the store’s price-matching policy.

If the advertiser doesn’t have brick and mortar stores, there’s a pretty good reason to be biased towards the negative assumptions. If you’re in that camp, it makes sense now for advertisers to parse out a duplicate campaign of top Google keywords to display on mobile devices while using the opt out option for existing campaigns.  The mobile campaign can be bid based on its own performance without diluting the results of the core program.

Mobile search growth is unlikely to slow anytime soon, while the lines between devices will continue to blur as more and more gadgets incorporate full web browsing functionality.  Google is clearly anticipating this as it pushes to get Android, with its close integration of Google services, on as many devices as possible.  Whether user behavior and our ability to track it begins to match what we see on traditional computers anytime soon is still an open question.

Technorati Tags: android, Google, iphone, market-share, mobile, paid-search, PPC, SEM, Yahoo

• eBay Adopting Syndicated Google Paid Search Ads, Dropping Yahoo Has eBay been phasing out syndicated Yahoo ads in favor of Google's? RKG records indicate a major change in the...
• Tale of the Tape: November Paid Search The numbers are in, and show some intriguing trends....
• Evaluating a Paid Search Program A layered approach to assessing paid search effectiveness. Now is the time to raise the bar....

A few years ago, I wrote a blog post referring to the syndication networks as “Sin-dication“. I’m trying to be nicer these days as a tribute to Alan.

FACT: The quality of traffic coming from the engine itself is of significantly higher quality than the traffic coming from their syndicate networks.

This has two important consequences:

1. Bidding the same for traffic on the Google syndication networks as on Google.com leaves opportunity on the table.
2. By not allowing advertisers to differentiate bids based on the quality of traffic driven by each network partner, Yahoo and Google lose money as do the advertisers.

THEORY

Let’s take some basic paid search principles and show how they apply to the syndicate partners. I’m going to make some simplifying assumptions to prevent us from getting mired in detail and missing the bigger picture.

The value of traffic in paid search varies terrifically by the user’s search and the ad it fires. Since we can’t really bid based on user search, let’s see what that looks like as a function of keyword:

Cool! Now let’s say the advertiser is willing to put 80% of the true value of the paid search income back into the marketing to drive the top line but still make some immediate profit.

Ideally, the bids would look like this:

By matching the bids to the value of the traffic on each KW we generate as much traffic as we can afford from each KW, thereby maximizing revenue within our efficiency requirements.

There is, however, another way to achieve the efficiency objective. Namely, by bidding to the average value of the collection of KWs.

In this simplest of models, assuming that the underbidding and overbidding don’t move the average value of the traffic — a big assumption and probably never the case — you still end up spending 80% of the paid search income on marketing, which was the target all along.

So if either way you hit the desired efficiency target, what’s the difference?

Volume. By wasting money on some terms, and missing opportunities for more sales on others, the poor bidding methodology hits its efficiency objectives but does so at the expense of sales volume.

Importantly: lower sales volume at the same efficiency by definition means lower advertising revenues for the engines.

APPLYING THEORY TO THE SYNDICATE NETWORKS

So, if what we’ve learned is that applying the same bids to traffic of varying quality leads to fewer sales/leads for the advertiser and less revenue for the engines how does that effect our view of the syndicate partners?

Let’s look at some data. I grabbed data from a handful of our clients and took median values by referring domain for click volume, and order volume as a percentage of the total traffic for each client. I also calculated median values for how each domain’s conversion rate compared to the average conversion rate for that ad network. The comparative conversion rate data is most interesting.

{Note: sites are ranked from left to right in descending order of traffic volume for the median client}

Notice that traffic from Google.com, AOL, Amazon, and Comcast tend to send much higher than average quality traffic: 16 – 20% better than average. At the same time, eBay, Ask and the Comparison Shopping Engines, tend to send significantly lower than average quality traffic.

For Adwords advertisers can and should bid differently on the Google.com domain and the rest of the network. Looking at these medians as if they were one advertiser’s data, we’d end up bidding the Google.com-only version of the account up by 16% and the Google.com + syndication network version of the account down by 33% (Google.com traffic is slightly more than 2/3 of the total, hence the disparity).

My sources tell me we’re among the very few folks in the space doing this, which is surprising as we’ve been doing it and advocating it to others for more than two and a half years.

This two-tier approach will help materially, but is in no way ideal bidding. In this, we end up underbidding on AOL and Amazon traffic, and still overbid on eBay, Ask and others. Better, but not great.

If we take a look at the same type of data from the Yahoo network we see even greater disparities.

Yahoo.com traffic is 35% better than the average Yahoo ad network traffic!!! The rest of the network averages ~ 42% below the aggregate average! But, even among those there are winners and losers.

Yahoo does not give us the opportunity to bid differently on the network, but does give us the opportunity to exclude traffic from particular domains. Given the lower traffic volume on Yahoo, it can be difficult to separate signal from noise to identify those domains that send particularly low quality traffic. Sometimes looking across multiple accounts helps us spot trends that we can’t really see looking at a single account’s data.

But excluding sites isn’t really what we’re after either. In most cases the traffic isn’t worth nothing, it’s just worth less. Paying the right amount for it would allow us to generate sales cost effectively from each domain in the syndicate network and most importantly, would allow us to push the gas harder on the high quality traffic provided by some.

Both Google and Yahoo claim to discount the CPCs from network partners, but our experience suggests the discounts aren’t adequate. Moreover, since any good bidding system bases bids on expected revenues, rather than sunk costs, the discounted CPCs wouldn’t solve the problem even if they were right. The bids might be right for the bad performers because of the discounts, but we’d still under bid on the higher quality traffic.

The obvious best solution for the advertisers and the engines is to allow us to bid differently by domains. Creating separate campaigns for each domain might be too much to manage, but account level percentage adjustments based on each advertisers data would be easy.

Perhaps the hang up is the publishers. If eBay’s revenue as a syndicate partner dropped by 40% they might use that space for display ads rather than search, and perhaps they’d rent that space through a non-Google exchange. Legacy revenue sharing agreements between the engines and the network partners may also be a barrier.

Whatever the case, we at RKG would like to have more control over what we pay for the traffic we get from the syndicate partners. We hope you folks will join us in this call!

This is one of those questions that has crossed my mind a number of times, but I never got around to looking at the data until now. I have no doubt that there are comScore studies out there, but with all due respect to those sharp folks, I don’t think their sample is representative of the general population.

METHODOLOGY

We took a random sample of our clients, and studied just over 1.8 million orders placed through our ads on those clients’ sites. We wanted to find out how loyal those buyers were to the engines they used. We eliminated all clicks from content networks, comparison shopping engines and other programs, so that we were left with only Google’s Adwords Network, Yahoo’s Network, MSN/Bing, and Ask.com’s Network. We also eliminated clicks on ads for different clients, and clicks more than 60 days prior to the order.

With this streamlined data set we answered the question:

For each engine, what fraction of the people using that engine first used that engine exclusively during their browsing?

We also looked to see whether the data looks different before Bing launched and subsequent to its launch.

FINDINGS

At first blush, the engines all appear to have fairly loyal followings.

This says that Google enjoys the highest level of loyalty with 97.6% of buyers who first touched a Google ad using only Google ads prior to their purchase.

However, this probably isn’t the right way to think about the phenomena. As we’ve noted previously, most PPC buyers touch only one ad before buying. The fact that they used only that one engine is perhaps less compelling evidence of ‘loyalty’ than we would see if we take a different tack.

In this next pass, we eliminated all the orders placed by people who only clicked one ad one time. What’s left answers the question:

For people who clicked more than one ad before buying, what fraction used the same engine/network for each click? And, has this changed since Bing launched?

These data paint a much more dramatic picture.

• Pretty clearly, Google users are much more loyal to their engine than the users of Yahoo and Bing.
• The degree of loyalty hasn’t shifted much since Bing’s launch, and one might surmise that Google folks are the ones testing the waters a bit as their loyalty number seems to have dipped a tiny bit.
• The Ask.com numbers are distorted by two factors: 1) on Ask.com Google ads intermix with their own; and 2) Most Ask.com traffic comes from their network, not Ask.com. As such, I’m eliminating them from the rest of the discussion (you’ll note some of the percentages below won’t quite add up to 100% as a result of excluding them after running the numbers.)

Once they leave, where do they go?

To do this analysis correctly, one would need to do a Markov chain analysis or something like it to look at transition probabilities. I didn’t have time to do that. Instead, I just looked at a gross measure of first engines and last engines within the set of multiple ad touch buyers. This obviates the distinction between Martha who went from Google => Yahoo => Google, and Ben who uses only Google. As such, having first engine and last engine the same is not identical to being an engine loyalist as studied above.

This is pretty cool data even given the crudeness of the study.

At first blush one might conclude that Yahoo and MSN are in big trouble: not only are Google users more loyal, but users of other engines are much more likely to move to Google than to any other. However, percentages lie. Google is the biggest so it’s normal that, for the subset that use more than one engine starting with MSN/Bing, most will go to Google.

The more interesting question is:

Do they migrate one way or another disproportionately to the size of the market share?

That’s what we show below. The expected percentage is calculated taking the relative market share — as defined by the entire data set from the first pass — of the other engines into account. We then calculated the percentage difference from the expected values.

This is kind of interesting! What strikes me is that MSN/Bing users are disproportionately more likely to move to Yahoo than to Google, and Yahoo users are disproportionately more likely to move to MSN/Bing. Perhaps this indicates that these folks really are committed to finding a Google alternative? I don’t know. I’m not entirely confident of my methodology/thought-process, and given that the first – last view is a pretty hacky approach we probably shouldn’t read too much into it.

CONCLUSIONS

Google appears to have by far the “stickiest” user base. This suggests that Bing has its work cut out for it if it is to gain share. That said, those who don’t use Google primarily are more likely to prefer other alternatives to Google than to follow the majority. The market demands a viable alternative.

WHO CARES?

This may be of most interest to the engines themselves — though they certainly already know this stuff — and perhaps to my friends at the big investment banks who want to gauge the future performance of those companies.

From a PPC management perspective, it’s kind of a non-issue. The lion’s share of the traffic comes from one touch buyers and engine loyalists. Moreover, by raw order count, the number of people who move from a first ad touch on engine X to a last ad touch on engine Y is almost exactly equal to the number moving from Y to X. This is true for all of the interaction Google <=> Yahoo; Yahoo <=> Bing; Bing <=> Google. As such the net effect on ad buying strategy is nil.

LIMITATIONS

Almost too many to list, but here are three more that might not be obvious.

• We’re only counting clicks on the advertisements. If someone clicks on a Google ad, then searches on Bing and clicks on a natural search link from that advertiser and makes a purchase, we’re calling them a Google-only user.
• Engine loyalty may have nothing to do with satisfaction levels. It is entirely possible that Yahoo and Bing are guaranteed to have lower loyalty numbers simply because of Google’s dominant market share. “Everyone” uses Google, so the curiosity-factor for Yahoo and Bing users has to be greater.
• We’re looking at Networks, here, so when Ebay switches from Yahoo’s network to Google’s people who used Ebay ads exclusively appear to have switched loyalties. Also, people who use off-beat search engines and bounce around might seem to be either loyalists if the sites all happen to be in one network, or switchers if they aren’t in the same network, even though their behavior is essentially the same.

I’d love to hear what others think about this. Am I looking at the numbers the right way? Am I thinking about the numbers the right way. Does this actually tell a story?

Technorati Tags: Bing Market share, MSN Yahoo agreement, search engine market share

• Customer Loyalty: A Case Study Customer lifetime value comes from protecting your customer's interests....
• Paid Search Market Share by Engine: Yahoo Resurgent? We took a look at "Market Share" of the big three engines over time and saw some interesting trends....

Over the past several months, our logs show that eBay had been testing Google ads for about 1% of its search ads, while the rest were being served by Yahoo.  Around 8/26, eBay flipped the switch and we began seeing the bulk of its clicks coming from Google ads.  Early this week we were still seeing eBay serve Yahoo paid search ads, but Google’s ads were providing roughly 90% of eBay’s search ad clicks.

While this move will likely benefit Google and eBay in the short-term, it is not necessarily a welcome change for advertisers.  We’ve noted before that syndication partners often have significantly lower conversion rates than the main engine providing their ads.  For many of our clients, eBay is no exception and we have often chosen to block our Yahoo ads from showing on eBay entirely.  Unfortunately, despite years of advertiser requests, Google does not provide the ability to block individual search partners in this manner.

Certain retailers are going to be hurt by this move more than others.  If your product offerings and keyword selection are particularly “eBay friendly” and you were previously blocking these ads on Yahoo, you could see a significant jump in costs without an equally significant increase in sales.  eBay has a ton of traffic and some retailers could easily see 5-10% of their paid clicks coming from the auction site.  We have been told that search network traffic is discounted based on quality, but that knowledge is not particularly helpful without more detail or the ability to make targeted search network exclusions.

As we’ve argued before, it is ultimately in the engines’ best interests to provide greater control and transparency to advertisers.  While RKG has tactics in place to handle the distinctions between Google and its search network, a perfect solution will not exist until greater control is available.  How many advertisers will decide not to bother with the search network at all if eBay starts eating up a big chunk of costs without delivering the sales?

Technorati Tags: ebay, Google, paid-search, PPC, search network, SEM, syndication partners, Yahoo

• Paid Search Market Share by Engine: Yahoo Resurgent? We took a look at "Market Share" of the big three engines over time and saw some interesting trends....
• Mobile Paid Search Data: iPhone Dominant, Android Rising Traffic from mobile devices has grown tremendously over the last year, but the quality of that traffic may be sketchy....
• Yahoo Q4 ‘09 Financials and PPC Share Yahoo! announced their Q4 '09 earnings on Tuesday, making their best effort to portray a 4% year over year decline...

Search Engine Land has a good write-up outlining the arrangement.

Our take? This is a good thing for PPC on three fronts:

1. The Yahoo service reps stay on using the MSN platform. This takes the best of both worlds. Yahoo’s reps have always been top drawer in our experience and the Bing API interface is excellent — it would be great if they didn’t update the API every time there is a .net update, and if they didn’t force ordering on XML which should be stateless, but…
2. By combining traffic, the Bing and Yahoo data sets will be far more robust allowing data-driven firms to work more of their magic. Heck demographic targeting might actually become valuable when there is enough data to model.
3. Reducing redundancy won’t hurt. Having to launch promotional copy across three engines takes time. Dropping from 3 to 2 will reduce grunt work and allow our analysts more time to focus on higher value analysis.

None of this will happen over night. They expect to close the deal in early 2010 after careful sniffing by regulators, and “full implementation” may take 24 months.

Will “Ya-Bing!” pose a real threat to Google? We’ll see. Looking just at Paid Search, the share is still only 20%. Google is still hugely dominant. But it certainly gives Microsoft a fighting chance. Now, if they can just pry AOL away from Google…

Technorati Tags: MSN Yahoo agreement, Yahoo Bing alliance

You may have heard hyperventilation to wit: Bing overtakes Yahoo! That turns out to be a gross overstatement, and a marvelous example of why it’s dangerous to extrapolate from small sample sets.

Here’s what we’ve seen so far in terms of PPC ads, the financial engine that drives the engines.

Remembering that Bing went live at the very end of May, beginning of June there are a couple of really interesting trends that leap off the page.

1. Totally unrelated to the Bing launch: Check out the weekend effect! Again, this is a percentage of totals, not a measure of absolute volume. It suggests that MSN gets a materially larger share of traffic during the work week than it does on the weekend. Does this mean people use MSN more frequently at work because it’s the default search engine for IE? Left to their own devises on the weekends, they choose Google/Yahoo even more often? Fascinating!
2. The Percentage of Total PPC Sales from the Big three coming through Bing increased substantially over MSN live, much more so than did costs, clicks or impressions. This could indicate better targeting logic, or that the folks giving Bing a test drive are decidedly higher quality prospects.

If we look more carefully into who’s losing the share of sales that Bing appears to have gained the answer is clearly not Yahoo. In fact Yahoo and MSN both seem to have benefited from MSN’s PR campaign. Not surprisingly the engine of choice for ‘early adopters’ — Google — is the one seeing its user base sniff out the competition.

With respect to Bing, the initial uptick in the quality of traffic is quite encouraging. Higher Sales per Click will lead to higher bids and more monetary share for MSN. The fact that despite the media blitz surrounding the launch, the impression share and click traffic share hardly moved is somewhat more concerning. If consumers are truly happier buying through Bing it could give MSN some momentum, but they can’t just rely on the early adopters and search geeks. They need to pull in and keep the average Joe as well.

As Lance Loveday points out, Bing has a steep hill to climb and it’s not clear that the media approach they’ve taken is going to do the job.

We’ll keep an eye on the trends. MSN has finally given us something to write about!

A follow up post, eliminating trademark search and controlling for a couple other factors paints a very different picture.

Technorati Tags: Bing Launch, Bing Market share, Bing Yahoo Market share, Google

• Bing: A Closer Look at the Numbers A more careful study yields a different perspective....
• Bing: The Art of Buying Share It's hard to buy loyalty....
• Bing Surges: Google buys TV ads Bing came out of the gates hard in 2010!...

Does your PPC program drive more sales than it’s credited for? The answer is certainly “yes”, but the sources and volume of under-reporting might surprise you.

If you drive your program through your web analytics software you may be missing 10 – 30% of the sales! This is not the fault of the software as much as it is the convenience of using javascript tags for tracking. In past years, part of the problem came from browsers not happy about running 3rd party javascripts, but those problems have been fixed.

The problem now is not with javascript flakiness as much as when the script is run. Almost everyone wisely puts the javascript tracking in the footer of their web pages. The reason? You don’t want the customer to have to wait for the javascript to run before loading the rest of the page. If there is any problem, you want the page to load and the footer to hang so that the user can shop unimpeded.

Herein lies the rub. Because it’s in the footer the javascript can only cookie the browser after the whole page loads. For heavy, slow, image-laden pages, customers often move on to the next page before the footer loads. If the user sits on that subsequent page long enough to fire the javascript the problem will be that you’ve lost the url parameters that allow your tracking system to know the source of the traffic. That user is now flagged as “untracked” even though they came through a paid advertisement.

We know this happens and understand the scale of the problem because it shows up whenever we do data audits with clients, but also because we sometimes employ both our standard tracking and a javascript tag when we’re studying marketing channel allocation for our PPC clients. The problem isn’t that orders we see are tracked to other programs; it’s that the sales we know came through a PPC ad aren’t tracked to any marketing program.

Knowing this, you might say: “Well, if I have a sense that this happens 20% of the time, can’t I just adjust my advertising efficiency thresholds by 20% to compensate?” Yes, you can, but the problem is that some destination pages are more susceptible to this problem than others, either because they load more slowly, or because users are more likely to navigate off of them quickly. This will disproportionately penalize some types of keywords over others resulting in lost opportunity as those ads are mistakenly bid down the page.

A better way to track high-dollar marketing programs is through use of a fast redirect. The redirect is fast if, and only if the redirect server doesn’t have to do a database look-up. If the server has to look up the destination url the redirect will be slow and the server will bog down during traffic bursts. We pass the final destination url to our redirector as an encoded parameter so the redirect takes less than 0.1 seconds and the volume of redirects is almost irrelevant.

Using a redirector provides much more robust tracking, but can/should be cause for concern as well. With all of that valuable traffic passing through a third party box it’s valid, indeed essential to ask: “what happens if the box goes down?” We stressed out about this, too. Our approach was to build in multiple redundancy by having multiple redirectors. To keep these independent, these servers are located across the country, and use different internet backbones. All the servers share the work, and are self-checking and self-correcting. If a data center becomes unavailable — say, due to a server failure (almost never), or due to DC connectivity problems (very rare), or due to routing hiccups somewhere on the web (not rare, but brief) — we use smart DNS to reroute traffic to the healthy machines within a minute. No system is 100% perfect, but this redundancy and automatic checking provides extremely high uptime.

Handling order allocation issues can be done on the fly, with the confirmation page tag sending marketing allocation along with the order details, or through a back-feed of order ids and marketing channel credited. Any competent search agency can then base its bidding and reporting on only those sales your system has flagged as paid search or unknown.

We see all the costs associated with PPC advertising, but we don’t see all the sales generated. While this post may cause some eyes to glaze over, understand that this is not minutia by any stretch of the imagination.

Better tracking technology plugs a big hole, but others remain. Users drop cookies, use multiple browsers, and sometimes search on one machine but place the order on another after shopping around. We can track spillover to the call center, but measuring foot traffic driven to the stores remains elusive. However, those who throw up their hands and conclude that direct marketing metrics shouldn’t be applied to search simply aren’t trying hard enough.

Technorati Tags: javascript, measuring SEM results, PPC tracking, web-analytics