Site icon Search Engine People Blog

Sitemap autodiscovery for the Big 4 via robots.txt

As Yahoo, Ask, and MSN have all reported this morning, they (and Google) have joined together to make sitemaps autodiscoverable via the robots.txt file. As long as you tell the search engines where your sitemap lives (in the robots.txt file), they will be able to find it and use it. Simply add the following line to your robots.txt file:

Sitemap: <sitemap_location>

The <sitemap_location> should be the complete URL to the Sitemap, such as: http://www.example.com/sitemap.xml

This directive is independent of the user-agent line, so it doesn't matter where you place it in your file. If you have a Sitemap index file, you can include the location of just that file. You don't need to list each individual Sitemap listed in the index file.

You can get all the facts about this and everything you need to know about sitemaps at sitemaps.org.