The Rimm-Kaufman Group launched a study about a month ago to try to determine the impact of geography and demographics on PPC user behavior. We theorized that just as catalogers know that certain regions of the country and certain specific zip codes respond to catalog offers at a much greater rate than average, perhaps we'd see similar behavior in Paid Search.
We anticipated that areas with low population densities would have fewer brick-and-mortar stores and that people using paid search links would therefore be more likely to be buy than average. We expected traffic value to be also strongly linked to wealth.
Digital Element was kind enough to provide their highly accurate mapping of IP addresses to zip codes for two of our larger clients: one an online pure-play, the other a major brick and mortar chain. We are enormously grateful to Digital Element as their data played a critical role in the study. We tried this previously with a cheap database purporting to do the same mapping and found that data to be unreliable and almost unusable as it was formatted.
Using the Digital Element data we tied about 3 million PPC clicks to their zip codes for one data set and almost 2 million click-throughs for the other. We then tied each zip code to census data we purchased which provided information about the location, average household income, population, land area, average age, average home value, number of business addresses, number of residential addresses, etc.
From this and our own conversion tracking, we were able to determine the impact on traffic value of each of those factors independently and in combination with other factors. We hoped to find a model that yielded strong correlation, allowing us to bid differently by zip code, better marrying bids to the anticipated value of the traffic.
We used the open source stats package "R", many different statistical modeling techniques, several different data representations -- by individual zip codes, and by data aggregated to zip3 and zip2 where necessary for data accumulation (many thanks to Kevin Hillstrom for that idea!); we looked at logarithms, squares and inverses of the values -- to determine the best model.
There is very little correlation between any of these factors, alone or in combination, on traffic value in the cases we studied. Our best, reasonably simple, model used the combination of average home value and population within the zip code. However, this model had an adjusted R-squared value of 0.15, meaning only 15% of the variance in performance could be explained by the model, the other 85% was some combination of random noise or other factors not available in our data.
There were a handful of zip codes and regions that appeared materially better than average, but on balance, we have to conclude that knowing the zip code of the user doesn't have much value for retailers in Paid Search bid management. It may be much more predictive for financial institutions trying to anticipate lead values, and it may have tremendous value for advertisers to have this information on their websites to target offers appropriately, note store locations and local offers, etc. But for national retailers, it appears that the person in rural Arizona searching for a leather ottoman is no more, or less likely to buy it than the person in downtown Chicago.*
We did find that the folks in rural areas were much morelikely to conduct searches than average. Taking the volume of clicks and dividing by the population within the zip code, we found that nearly 45% of the variance in clicks per person could be explained by population density.
People in rural areas and densely populated urban areas are more likely to use PPC ads than folks in suburbia and small towns, but by and large they're no more likely to buy when they do.
As a physics student in college, my favorite physicist was Johannes Kepler. Kepler developed an incredibly complex model to explain planetary motion. The test for any such model was whether it could accurate predict the position of planets in the future. Using the new, improve telescope designed by Galileo, Kepler found that his model was off by 2%. Just 2%! But that 2% was enough to convince him that his model was wrong. He scrapped all that previous work and went back to the drawing board, eventually determining that the planets followed simple elliptical orbits.
Kepler had the integrity to reject his own theory as a first step towards getting at the truth. We're no Keplers, but we can and will admit when we're wrong.
"So, if you didn't find anything, why is there a Part 2 post?!?" A reasonable question.
In the other piece of the study we looked at the impact of having a physical store in the zip code of searchers on both their propensity to search and their conversion rates. What we found was both fascinating and quite the opposite of what one might expect. Stay tuned!
*We're not saying that geo-targeting doesn't make sense in many cases. It absolutely does. For regional chains conversion rates are often better in areas where the brand is known and trusted; for local service businesses it's essential. However, as my friend and client Eric Nadler put it: "If some fat, 55 year-old, man searches for "ballet slippers" I want to serve him an ad! He might have grand-daughters!" In many respects the act of searching takes care of the demographic/psychographic profiles for you.