Not infrequently, we’ve seen such “tools” produce dangerously off-topic, ineffective and costly advertising.

From George’s post:

While a controlled use of tools is valuable to build out and maintain the long tail, there is no substitute for human understanding in the more general, higher traffic category and subcategory terms.

Machines leave gaping holes and invariably use synonyms that are completely inappropriate - think about that “costume jewelry” category.

Cleaning this dangerous stuff out of a massive list takes more time that building the list carefully in the first place through machine assisted but human driven processes.

Full SEL article: Don’t Let Machines Write Your Keyword List

***Note on both graphs below, the clearance terms are based on a secondary y-axis for the purpose of scaling them to a level where trends can be observed***

That’s fine and dandy, but what we really want to know is if our customer’s are still buying, and if so, what types of terms are closing the sale? I pulled sales data for the same time period above and confirmed that increased traffic on clearance terms is, in fact, converting for us. We’ve seen three times more sales on clearance terms so far in 2008 than we saw in all of 2007, including the holidays. The only question left now is: what are they buying when they are looking for a deal? I compared the AOV of general terms thus far in 2008 to the AOV of the clearance terms over the same period. No big surprise here – AOV of the clearance terms is 12% lower than the general terms.

People are still shopping for deals and buying high ticket items during these tough times; they are just trading down for less expensive goods. If you’re managing a search program, be sure to:

• Keep a close eye on AOV trends and on clusters of terms that may be performing better than usual due to current state of the economy.
• Make deals easy to find on your website – include a sales tab/link that is highlighted to stand out, make offers prominent on the homepage, and be sure to have a smart search results pages for sale/clearance terms.
• Plan offers to raise not lower AOV - incent customers to add extra items to reach an additional discount tier - but be sure that the tier is about twice your AOV.
• Set PPC bids intelligently on your discounted terms - consider margin-based bidding or defining different econ targets for specific clusters of terms.

We do this for our clients when appropriate; you should make sure whoever is managing your PPC campaigns does this as well.

RKGDuck slides a snippet of perl (or a big honking mass of perl, should you so need) into the Windows cut-and-paste clipboard buffer. This lets you do powerful text transformations within and between all Windows apps, using the familiar Control-C (copy) and Control-V (paste) mechanism.

After cajoling your local perl geek to write a tiny filter for your task, you can share that functionality easily with less-technical users, and you don’t have to install perl on the end-user machines.

Here’s a new RKGDuck video showing how in a couple minutes you can write a powerful filter to help clean up keyword lists.

You can watch other how-to videos and download the app itself on the RkGDuck homepage.

RKG provides RKGDuck free to the community under GNU General Public License. RKGDuck us distributed in the hope that it will be useful, but without any warranty; without even the implied warranty of merchantibility or fitness for a particular purpose.

If you’ve found RKGDuck helpful and can share your experience, or if you’ve hit any snags, do comment below.

Quack!

I received a curious piece of spam this week. It merits comment as it illustrates a critical concept in direct marketing: list-offer-package. The L-O-P concept also applies to running paid search marketing effectively.

Here’s the spam:

—–Original Message—–
From: gsemlinger@famfontana.com
Sent: Tuesday, June 17, 2008 8:24 AM
Subject: Mar’ket wacth

Now -you can see what we ” were saying.

Fir,m: An gs trom Mico syste ms
Buy as: amgs.ob
Strongly R ec ommend
Rece”nt: .4_0
Vloume: 331,48-5

Atfer the gerat news last week, volumed traded hit 331,485.

Mroe great things to come, t,he cli_ent base is growing and production is boimong A-G,MS is the one to watch.

Stlil a low investment thresh.ol”d, purchase Ang strom in the morning on Teusday.

OK, this is clearly a pump-and-dump stock scam for a penny stock, AGMS.OB. But to defeat Bayesian spam filters, Gsemlinger ran the email through some sort of obfuscation filter, which sometimes tossed in extraneous punctuation (Rece”nt) and sometimes permuted some letters (stlil, Teusday, gerat, boimong).

You can still make sense of the email, but it reads as if it was composed by a moron. Seriously, who in their right mind would take stock investment tips from anyone who writes that poorly?

Hold a sec. Let’s assume that SpamBoy Gsemlinger isn’t stupid. Let’s examine his marketing strategy.

Gsemlinger consciously degraded his response rate (the typos make humans less likely to act on his hot stock tip) in order to increase his list size (the typos make Bayesian filters more likely to let the message pass).

Why?

Why not instead send a beautifully composed long email, effusively and compellingly touting Angstrom MicroSystems, the wonder stock?

Perhaps Gsemlinger knows the List-Offer-Package rule, or L-O-P. Offer trumps Package, and List trumps Offer. It is takes less effort to win by growing the universe than by growing the response rate.

What do I mean by L-O-P? In direct marketing in general and in paid search in specific, who you are talking to (List) matters more than the details of what you are selling (Offer), and how you present your offering (Package) is the least important factor.

This doesn’t mean direct response creative and copy aren’t important; they certainly are. But creative and copy are less important than the offer itself, than what you are selling — the price, quality, and uniqueness of your merchandise; your service, guarantee, support, and shipping policies, etc. And more important than that is the quantity (sheer numbers) and quality (relevance, targeting, prior purchase history) of the prospect universe you’re addressing.

George Michie hits the nail on the head when he discusses how paid search analysts should spend their time to get maximum benefit.

Truly horrible PPC copy can hurt you, yes, but there’s small incremental sales benefit going from “decent” copy to “great” copy (low single digit response lifts, if that).

Truly horrible PPC landing pages can hurt you, true, but the incremental benefit between “decent” and “great” landing pages is small (typically single digit response lifts, if changes are just design and layout).

By all means work on copy and landing pages, particularly if they’re sub-par. But place your emphasis on the size and quality of your marketing universe. It is far far easier to grow your List by 10% or 20% (perhaps even 50%). If you can keep the quality up, increasing the number of individuals to whom you’re marketing increases your sales proportionally, and your profits hyperproportionally. Dollar for dollar and hour for hour, increasing the number of quality prospects with whom you’re speaking is the most cost-effective and time-effective marketing lever you can pull.

Gsemlinger faced an either/or tradeoff, where he opted to buy List (circ) at the cost of Package (response).

Happily, in paid search, we don’t need to degrade our ads to get more volume. In PPC, getting more qualified traffic translates to compulsively-extensive keyword lists, judicious use of match types, and razor-sharp economic-based statistical bidding algorithms.

Keywords, bids, and matchtypes — those are the big levers of paid search. To get your PPC programs generating maximum profit, focus there first.

Or as our friend Gsemlinger would advise: Keywodrs, bids, and match”types — those ar-e the big levesr of pa,id serach. To get your PP-C program,s genertaing maximu.m profit, focus tehre first.

Back in April, Yahoo began a test with Google running Google ads on something like 3% of Yahoo impressions. This was undoubtedly a “proof of concept” test, and apparently both parties were pleased with the results. Why wouldn’t Google be pleased? They picked up additional revenue simply by adding another, more substantial than average, syndication partner for its “AdSense for Search” product. Yahoo clearly made more money serving Google ads than it made serving its own.

If federal regulators let this happen – a pretty big “if” – Google and Yahoo both win, and Microsoft would lose out. Yahoo will have successfully fled from a hostile takeover bid by Microsoft by throwing itself into Google’s arms.

But what about the advertisers?

Danny Sullivan asked me to opine on the impact of this, and the article is over on Search Engine Land.

Let me know what you think!

In general, many catalogers and retailers tend to place a higher value on orders recorded as new customer acquisitions. Traditionally, direct marketers have often been willing to lose money to acquire new customers up to a one-year value metric, while they expect to turn a profit on their existing customer base. How much you can afford to invest may depend on the lifetime value (LTV) calculations for each channel.

The first step in measuring the value of paid search orders against your other marketing initiatives is to perform detailed matchback analysis. Have your SEM agency provide a list of PPC order numbers taken over the last 90 days. From there, run an audit of those orders internally to determine if the customer placing the order was new to file or existing. You should now be able to send the list of orders back to your SEM agency for follow-up analysis. Reports can then be generated to show what percentage of customers are new to file in paid search, how that number differs across Brand and Non-Brand keywords, what types of keywords/categories drive more new customers, effects of seasonality/catalog drops/etc on new customer acquisitions – the list goes on and on.

Comparing these results to other marketing initiatives might shed some light on which channel may be most effective on the next dollar invested. It might be the case that catalog prospecting averages $20 per new customer acquisition, while non-brand paid search is hitting a $15 mark in aggregate. When deciding which channel should receive the next advertising dollar, it seems reasonable to pick the one with the lowest cost per acquisition. Ignoring averages and looking at the next dollar in, we may find that the incremental dollars given to catalog will produce one new customer acquisition for every $21 in spend and $25 in paid search. Look at each channel’s ability to capture new to file customers on the next dollar spent when deciding where to invest additional funds. Also, you’ll need to measure the one year value of non-brand PPC customers vs. those acquired from traditional marketing methods. Ultimately, it is these calculations that will determine what you can spend and, most importantly, what each channel’s acquisition targets should be.

An interesting question can be raised about the value of an order resulting from a non-brand keyword search. For example, an existing customer searching for “desktop computers” is bombarded with competitors’ ads along with your own. Is there incremental value in existing customer orders resulting from non-brand keyword searches? We think so. Remember that many of your loyal customers are also loyal customers of your competitors; you need to consider share of wallet here. (More on that, in a previous post). The matchback analysis is not an initiative to discredit keywords, or the program at large, for existing customer orders. The intent is to look at the overall influence of paid search in conjunction with your new customer acquisition goals.

We take our aggregate spend on Google, Yahoo, and MSN across all RKG clients and show how it breaks down by month, by engine. (Search and Content combined, but the data is heavy on Search) In addition, since advertising spend is set on economic goals and not engine favored, this approach removes search engine bias, seasonality and RKG growth. In other words, the allocation of spend reflects availability of quality clicks on each engine, and we’re using relative proportions.

May 2008 results: steady, no big change.

Google @ 80.8%
Yahoo @ 14.5%
MSN @ 4.7%

Google’s at ~81% because their clicks work. As for market share, no big change YTD.

Here’s a glimpse at CPCs across our client base for the month.

The results: Google earns more per click.

Next, CPC trend by month, indexed so google = 100%. (This removes seasonality and RKG growth.)

Pretty stable. Google consistently has 10 to 20% higher CPC than Yahoo, MSN.

CPM (cost per thouand imps) for May across our clients:

MSN actually earns more per imp than Google, though on far far far fewer imps and clicks (our clients in aggregate bought 14 times as many clicks on Google vs MSN in May).

Trend in eCPM, indexed so G = 100% to remove seasonality and RKG growth:

Reveals that as we buy more traffic from MSN, we see eCPM decreasing.

Stay tuned for next month’s post!

APT provides you more information about your search campaigns than you’d have otherwise. But some advertisers mistakenly believe APT reveals more than it actually does.

I’d argue the problem doesn’t rest with Google, but rather with advertisers’ mental models of search results pages (SERP s).

OK. APT lets you see a “clean” search results page for a given keyword, Google domain, display language, country, and geography. For example, say you are a US Google advertiser based in NYC. APT lets you check out your local ads in different regions nationwide without leaving your desk.

Some paid search advertisers (or their managers) search on their own company’s terms regularly (sometimes obsessively) to check the position of their ad relative to their competition. Google does report daily average position data, but that dry stat doesn’t reveal competitors’ ranks, and doesn’t provide the visceral satisfaction of “making sure myself that our ads are where they should be” by pulling up the “actual” SERP.

But manually searching your own terms from one computer to check your own ads may not tell you the full story.

I mentioned the geography issue above — sitting in NYC, you shouldn’t (and wont) see your San Francisco campaigns.

Also, Google adjusts ad serving based on IP click-through rates. If a given IP or (perhaps even user) shows a statistically significant anomalously high or low CTR for a given ad, Google may stop serving it.

An example: a major retailer with a strong focus in the (sorry for the obfu) widget category spends large sums advertising the single word term “widget” on Google. As a result, their managers often search for “widget” to check their ad and their competitors’ ads. Obviously, they seldom click their own ad. Because the CTR on this high-impression ad is effectively zero from within their corporate IP range, Google stopped showing their ad to searchers at the corporate HQ building. This caused some valid concern (”Why aren’t we showing up?!?”) until explained.

Enter the Ad Preview Tool.

“Hallelujah !”, advertisers rejoiced, “a way for us to get to the ‘true’ SERPs, cleansed of geo effects and individual history effects and all other Google muck. Like the platonic simplicity of GoTo’s CPC-displayed-on-SERP approach. Now we can keep an eye on on ads’ true rank (and adjust bids to hang in position three). Hallelujah!”

Beyond the parenthetical bid-to-position stupidity (B2P nearly always trashes advertiser profits ), the problem with the prior paragraph is the notion of a true or pure or absolute SERP.

Because in 2008, my SERPs often aren’t your SERPs.

Google knows too much about me. Where I live. What feeds I read. What I’ve searched on before. What certain terms mean to me. Fast: what is a bass? For you, fish or instrument?

Google also varies SERPs to test new ads. Google burns a small fraction of impressions “over-serving” baby ads so as to determine their QS to price ads “correctly”. I’d wager this sampling is essentially random over time, users, and data centers. Again, this leads to some differences in SERPs between users.

Which brings me to my main point:

There is no such thing as a platonic or “true” universal SERP for a given phrase.

SERPs vary to some degree by user, by time of day, day of week, by geography, by IP, by Google data center, by user search history. Who knows, maybe by Google market cap and phase of the moon. What is certain is that there are many many factors involved.

Sure, if you set an outrageously high maxbid, say $25 or $50 for “widget” (this time not obfu, just generic) and don’t set a campaign budget cap and do have a well-known brand name (thus high CTR) and do have OK ad copy and do have fair prices on your merchandise and and decent selection and do have reasonably usable site, then yes, with very high probability you’ll effectively “own” position #1. And that would hold for just every user and every SERP in your geography. (Dell takes this approach for many computer terms.)

Dominating the top may or may not make economic sense, may or may not make branding sense, but you can get there if you want. Just spend enough. OK. In the case of one advertiser determined to outspend all others, yes, the “true” SERP will always have big-spending-advertiser in the first position.

But that atypical. Usually several savvy big-spending advertisers jointly share the top of the page, jostling up and down a slot or two based on bidding algorithms, management ego, whatever. As positions vary by bidding and are filtered through user geography and user search history, there’s no “true” PPC SERP in this case. And I’ll argue that that is OK.

“But wait,” you may be saying, “doesn’t APT reveal the True SERP?”

I think not, as APT also filters on the user data (here “user” means APT user, the advertiser). Let’s watch this in action.

I turned off cookies (FFox: Tools >> Options >> Privacy >> Uncheck “Accept Cookies From Site” box), logged out of Google, deleted existing cookies (FFox: Tools >> Clear Private Data >> Check “Cookies” box >> Clear Private Data Now”) , closed all browser windows, and then opened a new browser. Clean as a whistle. I then went to https://adwords.google.com/select/AdTargetingPreviewTool directly.

In the tool, I specified “plumber” as the keyword, searching on “google.com”, language “English”, country “United States”, “All regions within this country”.

One would expect APT to return national search results for “plumber”. After all, that’s what I asked for, and there are no cookies involved. Here’s the result page.

Despite my request, the page is clearly geo-targeted for my town, beautiful Charlottesville VA. Half the paid ads are local ads: position 2, 4, 5, 7, and 8. So Google uses IP data, which we’d expect.

(Aside: I feel sorry for the guy in position 10 who missed the concept of local targeting. Errr, C’ville is 750 miles from Grand Rapids, friend. Here’s a sympathy backlink:Grand Rapids plumber )

How about trying APT with hiding our true IP? Turn off and purge cookies, close and restart browser, activate Tor , and back to APT

Hello from Austria! That’s where I came out of the onion, it seems. OK. Complete the form (”plumber”, “United States”, “google.com”, etc). and submit:

APT doesn’t like anonymous proxies. Banned! Couldn’t use APT for about a day. (Unless I was logged in with cookies on, then it would let me through.)

So let’s wrap this up. My suggestions:

• Don’t get hung up trying to determine the universal “true” SERP for a given search phrase. It doesn’t exist.
• Realize my SERP might differ a bit from your SERP for the same phrase at the same time.
• Realize even your SERP for the same same phrase might vary a bit when you re-query in a short interval.
• If you want to know where your ads are on the page, rely on Google’s average position stats. Averages are powerful summaries of the whole population of observations. Use them.
• If a term is performing above your economic hurdle and it isn’t already at the top of the page, spend more and move that ad up! (And then ask yourself why your bidding system allowed it to get out of position…)
• On the other hand, if a term is performing below your economic hurdle, lower your bid! (And again ask why your bid system overspent on it…)
• Do use APT to monitor your local campaigns in different geographies.
• Don’t use APT to monitor your national campaigns.
• To avoid getting banned, avoid the sneaky stuff (using automated scripts to scrape APT, using anonymizing proxies, etc)

Happy Monday, all!

Bonus tip: our Google reps confirmed that the natural results on APT are “authentic”, too. A small SEO tidbit there…

I’m speaking on this topic at SMX Advanced in Seattle next week, but I can’t go into much detail on that panel, so I’ll take the opportunity to flesh each point out here.

Bidding is second in importance only to an exhaustive, well thought-out keyword list for generating results in search. Contrary to what some would say, not all bidding systems are created equal, and the performance differences are material. If you’re in the market for a SEM let these “requirements” help guide your process.

Requirement #1: Proprietary System with Strong Statistical Foundation

You can’t be the leader in the space if you’re using someone else’s tool. Even if the tool you’re using is pretty good, the fact that you’re not elbow deep in the bowels of the thing means your flexibility is limited, your understanding of how it works is shallow, and you’re beholden to someone else for important new innovations.

Consider for example a tale of two keywords: Keyword 1 has a conversion rate of 5.0 % and an AOV of $30, for a sales per click value of $1.50. Keyword 2 has a conversion rate of 0.2 % and an AOV of $750. The sales per click value is identical, yet most bidding systems will have a hard time reaching that conclusion for keyword 2 consistently. Likely the calculated value will fluctuate wildly and its bids along with it.

We’re constantly making adjustments/improvements to how our system functions. This is a complicated game, the rules change, the objectives change, and we learn more and more as we go. If those changes don’t get folded into the system then you get trampled by the competition.

Some companies present their bidding system as “The Perfect Machine”, give it some cute trademarked name and tout it as the ultimate. I’m deeply suspicious of any firm that wouldn’t describe their system as a work in progress. Sure, we play the marketing game of throwing Alan’s PhD in Stats from MIT at folks and suggesting that our smart math guy is smarter than our competitor’s smart math guy, but the reality is that human behavior is hard to model, and what we’re talking about here is figuring out patterns in human behavior.

We’re still learning and innovating, but that’s what having a proprietary system is all about.

Stay tuned for more in the series, and swing by my panel discussion in Seattle if you get a chance!

George

From Techcrunch back in October:
Google assigns dollar value to search results

From SEL back in April:
Google Showing Ranking Scores On AdWords?

From Search Engine Journal, same event:
Google Leaks Quality Score Variables (Pscore, mCPC and thresh) in Search Results

Hattip: SEOBook