Each Element of a Search Ad is Ripe For Testing

Savvy direct marketers understand that ongoing testing is the key to improving results.

Done right, paid search advertising lends itself well to this rigor: Each element of a paid search ad- keyword, copy, and destination page- can be the basis of a profitable test regimen.

You’ll notice that many of the same best practices from offline testing apply online as well.

Test Shouts, Not Whispers. Test One Thing at a Time.

Testing takes effort, attention, and sometimes money. Don’t test tiny tweaks. Favor bold tests that have the potential to really change your marketing. Subtle tests will, at best, yield subtle results, often too small to detect.

Multivariate testing (aka scientific tests, design of experiments, or Taguchi testing) offers marketers the chance to vary many factors at once in a statistically valid way. We strongly recommend a walk-before-you-run approach. Start by changing one element at time, determining if that elements helps or hurts in isolation. After you have a strong testing program in place, you can later bring in a consultant or software to help with more advanced testing methods.

Make Sure Every Ad Has a Unique Tracking Code. Make Sure Your Tests Are Statistically Valid.

Each code should describe a distinct combination of phrase, search engine, destination url, and ad copy. Without accurate tracking, you can’t distinguish losers from winners. Detailed tracking also lets you “slice and dice” your results after the fact along different dimensions, gaining more insight from the same testing effort.

Make sure you can separate marketing signal from marketplace noise. Familiarize yourself with basic statistical significance calculations. As a very rough rule of thumb, if you plot conversion rates over time, a test needs to increase conversion by more than 1.5 times the normal range of variability to be significant.

Keep a Testing “Notebook

For each test, document what was tested, why, and what happened. Just a few brief sentences can prevent you from repeating mistakes or wasting time on questions that have already been answered.

Before the test, write down a clear hypothesis of what you’re trying to prove or disprove (”Test 6: Our hypothesis is that bringing visitors coming into our site from paid search to the new simplified product page template will increase conversions relative to the current grid-style product page template”).

Write down your decision metrics, and the roll-out plans. (”If the new pages increase closing by a significant amount, e.g. 50+ more orders than the control for the week, we’ll discard the grid template in favor of the simpler template.”)

After the test, record what actually happened. (”Despite one very large order, the simple treatment did not work, actually reducing conversion, but by a statistically insignificant amount. Next steps: keep the grid, test another challenger later this month.”)

Accept That Many Tests will be Duds

Be patient and be prepared for the fact that most tests produce a null result. And the more successful your program becomes, the harder it is to move the needle. Because your current marketing approach represents years or decades of thoughtful improvement, many alternatives won’t test out better.

Should this realism about the difficulty of hitting home runs dissuade you from testing?

Not at all. When you do test your way into an improved marketing strategy – be that better copy, better destination URL choice, or better landing page design – you’ve struck gold, more conversions for the same cost. While success may not come easily, the potential rewards are well worth it.

Get into a Testing Groove

As much as direct marketers recognize the value of testing, many admit that they test far less frequently than they think they should. Work with your team to establish a rhythm to your testing. Instead of one-offs driven by the latest buzz, establish a plan for a series of tests and stick to it.