Sometimes it’s tempting to leap to conclusions in paid search. In this cautionary post, we’ll provide a tool for testing whether data has changed meaningfully, or whether you’re looking at statistical noise.

Thin data impacts every paid search program.

In paid search, metrics often vary in unexpected ways. Unexplained variation is often referred to as “noise.” Noise occurs on top of, and can sometimes cloud our view of, the true underlying pattern, or, the “signal.”

As retailers, we always wanted to know: how are we doing relative to our competitors? We thought it might be useful to throw out some benchmarks of what we see across our client-base.