Nov 52008

3 Methods for Setting Cookie Windows

Someone clicks on an ad and visits your site. Some time later they buy from you. At what point should an ad stop receiving credit for 'driving' this order? In a subsequent post, we'll tackle the notion that "credit/no-credit" demarcations may not be sufficiently subtle, but for now, let's talk about ways to establish just such a line.

It's helpful to posit that with infinite cookie windows, each ad would be credited with some fraction of buyers who were driven by the ad and some who simply bought later without the ad having any influence over the purchase.

Perhaps the consumer in that later group is now on a different shopping mission, perhaps they've been influenced to visit your store again by other media or word of mouth, but particularly for well-known brands assuming that all orders preceded by an ad click were driven by that ad is hard to justify.

The other cohort of buyers were influenced by the ad. However, they may not convert same session because they want to shop around, they want to consult with someone (a spouse?) about a purchase before commiting, etc. Indeed, it's reasonable to assume that as the delay between the ad click and the order gets longer the split between the "considered buyers" influenced by the ad and the "random re-visitors" will change, shifting from heavily dominated by the former, to heavily dominated by the later as the delay gets larger.

This framework, if accepted, gives us three different mechanisms for determining the right cookie window.

Method #1: The Click-to-Order Curve

  1. Take non-brand PPC conversion data for the past two or three months. Brand traffic converts differently from non-brand and is generally not incremental, hence we recommend focusing on the competitive ad data.
  2. Count the number of orders placed within each day subsequent to the ad click that preceded the order. To avoid edge effects and rounding issues, we recommend classifying the click to order delay in seconds, dividing by 86,400 (the number of seconds in one day) and truncating the results rather than rounding, so that any order placed within 24 hours of the ad click is binned as a "0", meaning within the first day. A "1" is then between 1 and 2 days, etc.The data should look something like this (only showing the first 20 rows):

    Orders by Days Delayed from Click

  3. Make a column showing the running % of total orders, such that the number is always increasing towards 100%.
  4. Graph the day versus the percent complete as below, make sure all days are represented, filling in days with no orders as needed.

Cookie Window 2

The argument goes that if the data is dominated by the "random re-visitors" that will manifest itself in a straight line graph. Essentially, if there is no connection between the click on the ad and the order, then there should be no bias towards orders happening sooner rather than later so the "rise over the run" should be a constant.

When we look at data stretched over a long window (90 days or so), we often see a normal-looking curve followed by a long fairly straight line up towards 100%. We interpret this as showing the break between the buying cohorts being primarily composed of ad-driven traffic delayed for normal reasons, and the later flat line showing the traffic being primarily composed of random re-visitors not driven by the ad.

Find that point by tracing a line from the right side of the graph and see where the data truly dips below that line. That day is a reasonable day to define as your cookie window.

Cookie Window 3

Method #2: Average Order Size

  1. Using the data from Method 1 Step 2 calculate the Average Order Size for each day delayed. If the data is sparse you might want to take a three- or five-day rolling average to smooth out the spikes.
  2. Cookie Window 4

  3. Make a graph of Average Order Size by day

What you will often find is that the average order increases for a period of days then drops back closer to the overall average or same day average. We'd argue that this represents another flag indicating that the delay is no longer a function of more considered purchases for higher ticket sales, but is simply random repeat visits unconnected to the initial ad visit.

Cookie Window 5

Method #3: Keyword to Shopping Cart Match Rates

This is the hardest to pull off comprehensively, but can be done in an hour if you're willing to use sampling rather than complete data sets.

The notion is simply that usually the keywords searched bear some relationship to the items purchased. Not always, and the fact that sometimes a person goes to Land's End looking for a shirt and ends up with pants instead doesn't mean the shirt ad shouldn't get credit for bringing them in the store, but it's reasonable to suspect that the fraction of orders matching a keyword thematically should be reasonably constant.

However, the farther delayed the actual purchase is from the cookie, lower that match rate may become.

We recommend sniffing this landscape by

  1. Take 20 orders that closed the same day as the click and see what fraction of orders had items that matched the keyword. Exclude overly general keywords like "apparel" for since 100% of orders will match that keyword.
  2. Repeat this process with orders around the cookie window(s) suggested by Methods 1 and 2 above
  3. Repeat again with orders as far away from the ad click as you can find as a benchmark on the other end

The more samples you take the better sense you'll have of the landscape and the variability of the data, however if you're comfortable with "squishy" numbers this can help you determine if your cookie window makes sense and can confirm the numbers the other methods point to.

Proper attribution is critically important, and we recommend revisiting your cookie windows periodically to make sure you're in the right neighborhood. Too long and you're liable to waste money by over advertising, too short and you may miss opportunities.

Hope the above is useful and if people have other ideas let me know!


7 Responses to "3 Methods for Setting Cookie Windows"
Jim Novo says:
Nice work. We used a 4th method in the Lab Store (exotic pet supplies) by looking at the click to buy span of "head" phrases, e.g. "ferrets", that are early cycle search. We guessed phrases like "ferret supplies" and "ferret food" indicated a person closer to purchase. In fact, that's what happened. with "ferret" closing in 13-14 days, "ferret supplies" in 7-8 days, "ferret food" in 3-4 days. Not as scientific as fitting a curve, but works for a small biz like the Lab Store. Re: "Brand traffic is generally not incremental", I have always thought the underlying reason for this was "search for navigation". People are putting the Brand in the search box not because they are searching for info, but to navigate to the site instead of typing in a URL - it takes less time and the Brand in the first link in the SERPS 99% of the time.
Thanks Jim, your point is well taken. I'd argue that it in addition to the degree of specificity, there are different shopping patterns inherent in different sub-categories. The person searching for "guitar strings" is much less likely to shop around than the person searching for "guitar amps" for example. Partly a function of higher price tag, but there are other intangibles as well. However, having different windows for each keyword present a level of complexity even we shy away from! It could be done though. Indeed we actually recommend setting cookies that never expire and simply handling the "window" in reporting algorithms. Conceivably that algorithm could apply a different window to each word but we haven't gone there yet. George PS: I agree with your assessment on the brand traffic. Whatever made them search for you by name is what deserves the credit, not their path to your site.
andrea says:
All of this makes sense & is reasonably straightforward to pull off... The bigger issue that we are facing is not attribution of sales with one program based on cookie duration but attribution of sales across *multiple* online marketing programs, all of which are based on cookie duration and only some of which are based on % of sales attribution. For example, the generally accepted business model for affiliate marketing is to pay a commission on sales generated based on pixel tagging within a certain duration. For behavioral targeting, it may be to pay for a *percentage* of sales generated based on pixel tagging within a certain duration. When you add up all the online mktng programs across a company's portfolio, there is certainly risk of double counting & double paying but also risk of not understanding what truly happened--did they see an add and then type a keyword? Difficult to tell if you are only using cookie duration and pixel tags across various vendors. Solutions? Is there such a thing as a "universal" pixel tag that might track impressions/impression dates; clicks/click dates and orders/order dates across multiple programs. Do companies use their analytics solutions entirely to review & analyze this information? Offline matchback analysis also including housefile email & directmail is an option, but it is time consuming and still does not tell you which came first or how cost & sales should be attributed. Any thoughts? Any recommendations on where to start?
Andrea, thanks so much for your thoughtful commentary. You are absolutely right, the game is more complicated than current attribution schemes acknowledge. We're actually in the process of developing exactly this type of service to help retailers and others understand the interactions between channels. Our early research indicates that multichannel interaction is material, impacting 20% - 35% of online orders, and that allocation schemes have a tremendous impact on how retailers view channels: with some channels seeing big changes depending on whether first touch or last touch gets the credit. Many retailers are using rules based logic to spread credit between programs, but I think very few have conducted the research you're talking about to determine those rules, relying on gut instinct instead. I'm a big believer in gut instinct, but data is important too. We believe that the right way to do this analysis will involve a cookie that never expires, and a good deal of analysis on the propensity of some channels to initiate contact and others to cannibalize credit. Complicated stuff, but we like that kind of thing! George


Check out what others are saying...
[...] How do you determine the optimal “cookie window” for your ad campaigns? In other words, how long after someone clicks on an ad should that ad be credited for driving a sale? Alan Rimm-Kaufman explains 3 approaches to tackling this question. [...]
[...] Cookie Windows matter and should be carefully considered. [...]
[...] The greater the order latency, the bigger the impact. We typically find that more considered purchases, and higher AOV advertisers have greater latency than average which impacts the proper length for the cookie window. [...]

Leave A Comment