| Title: | Robots-Nocontent |
| URL: | http://www.rimmkaufman.com/rkgblog/2007/05/04/robots-nocontent/ |
| Printed: | October 12, 2008 |
| Source: | The Rimm-Kaufman Group Blog, info@rimmkaufman.com |
- May 4, 2007
- 1 comment
Yahoo has introduced a new tag, “Robots-Nocontent”, which allows web masters to keep the Yahoo spider from reading certain portions of a web page.
The robots.txt protocol has long allowed webmasters to keep spiders off entire pages.
This new tag gives webmasters intra-page exclusion control.
Webmasters can now mark parts of a page with a ‘robots-nocontent’ tag which will indicate to our crawler what parts of a page are unrelated to the main content and are only useful for visitors. We won’t use the terms contained in these special tagged sections as information for finding the page or for the abstract in the search results.
The most obvious use of this tag is wrapping navigational and menu elements. On many websites, identical navigation category links appear on every page. These links help users get around the site, but could confuse a spider about a page’s meaning.
The code is just a CSS class applied to a DIV or P or SPAN tag:
Because this paragraph is styled with the robots-nocontent class, the Yahoo spider should ignore it.
I can’t see any downside to this useful convention; hopefully Google will soon follow suit.
If you like this post, consider subscribing to our RSS feed. You can also have new posts sent to you via email.
Possibly Similar Posts
- Google’s Maile Ohye: 100 links per page, max
- Staples.com’s Robotic Warehouses
- Should You Sculpt Your Google PageRank Via Internal “No Follow” Links?
- Allurent, YesNoMayB, Shopify, Wufoo
- Controlling Your Message in Google Snippets Via NOODP Tag
I really enjoyed this post Alan. The new tag will be a useful tool and your post rewards those SEOs with the dedication to read up on new technologies on an ongoing basis. I’ve browsed around your blog and found plenty of other quality posts, so I thought I’d let you know that I like your blog. (On a related note, would you care to trade blogroll links? MontrealSEO.ca)
Regarding one of those other quality posts - #3 on pricing - while my own focus is on the SEO side of things, I’ve also had people approach me with a rev-share offer. Interesting to see the downside to that from a client point of view. The flip side is that for new brands and companies, that low-hanging fruit issue is not going to be as relevant.