How should travel sites such as hotels and airlines deal with the flood of site search pages generated when users either search for dates and locations, or do a general site search? While drowning your troubles like this guy might be tempting, it’s not so helpful. If you work in SEO for the travel industry, this is a question you’ve probably dealt with, and according to my research across a number of leading brands; there are a variety of ways these pages are being managed.
With Google’s recent tendency to return site search pages in the SERPs, it’s a topic worth serious consideration. For the travel industry, the issue I see with Google returning site search pages generated from date/location searches in the SERPs, is that rates and availability change so quickly that an indexed search page that's even 1 hour old could be useless because the flight or room category is now sold out, and/or the price has changed due to revenue management rules.
For my research I simply used the advanced search operators ‘site:’, ‘intitle:’ and ‘inurl:’, so the numbers are pretty rough, but close enough to make my point.
• Marriott has around 3,890,000 URLs indexed containing ‘search’, which includes a variety of types of search pages. Consider that the total number of pages indexed for this domain is about 5 million. So nearly 80% of indexed pages for this company are site search pages.
• IHG disallows URLs containing ‘/*/searchresult*’ or ‘/*/hotelsearchresult*’ in their robots.txt file, so they have no direct date/location search pages indexed (not including suppressed listings). However, they do have around 500 pages indexed containing ‘search’ that appear to be generated when multiple hotels match a search.
• Ritz Carlton has around 2,000 pages indexed with the URL containing ‘Reservations/Default.htm?ci=’, which is generated after searching for availability at a certain location.
• United.com: approximately 15,000 URLs indexed containing ‘search’, out of nearly 2.1 million total URLs indexed. Obviously a very low percentage. These are a mix of general site search, car search, cruise search, etc.
• Delta.com: uses a single URL while it searches for results, then dynamically loads those search results, appending the URL with ‘#top’ to return results, entirely avoiding the creation of new URLs. However, when looking for indexed pages containing ‘search’ in the title, Google returns about 6,500 results, many of which come from the Delta blog, SkyMiles Marketplace and various other international subdomains.
As you can see, approaches and results vary widely.
If our goals are to streamline the crawl experience for Googlebot and other search engine spiders, and to present well optimized, highly converting pages to users via the SERPs, we can do better than to leave all search result pages open to indexation. The question is, how?
Primary Strategies for Handling Search Pages
Here I’ll identify the primary methods of dealing with these pages:
• Disallow in robots.txt: Also known as the ‘sledgehammer’ approach, it might be a little more brute force than necessary. This method uses disallow statements to block all pages containing a certain phrase from being crawled and indexed. While effective in certain situations, the main issue in this case is you could be missing out on valuable traffic and revenue.
• Allow all to be indexed: This method results in a much higher number of total pages indexed than necessary, meaning the index is bloated with undesired, poorly optimized pages that don’t convert well if at all.
• Exclude via parameter handling in Google Webmaster Tools: This can be an effective way to handle dynamically generated pages, but it also has certain limitations. Accounting for all of the various permutations of URLs and parameters can be difficult and unwieldy, creating a nightmare to effectively manage. Even if a site has a clean URL structure, you’re potentially missing out on valuable traffic, which could be optimized for.
• Research and Optimize: We’ve learned from ecommerce that some search result pages can generate a good amount of traffic and conversions, so this option calls for us to research high performing search results pages, then create static, well optimized pages that draw traffic and convert well. Unwanted search result pages should be blocked from the index using a smart combination of meta robots ‘noindex, follow’ tags, and depending on the situation, carefully crafted disallow statements in the robots.txt file.
The last approach should result in well-optimized pages, which draw traffic in the SERPs, convert well and ultimately drive more revenue by providing users with the information they want.
Hopefully this has been helpful in thinking about how to handle site search pages. For more insight into SEO and PPC for travel, keep an eye out for upcoming blog posts from Adam Reitelbach outlining essential SEO and PPC tips for travel.
And if you’re attending the PhoCusWright conference in Scottsdale in November, be sure to come by and say hello.