Sitemap autodiscovery for the Big 4 via robots.txt

by Donna Fontenot April 11th, 2007 

As Yahoo, Ask, and MSN have all reported this morning, they (and Google) have joined together to make sitemaps autodiscoverable via the robots.txt file. As long as you tell the search engines where your sitemap lives (in the robots.txt file), they will be able to find it and use it. Simply add the following line to your robots.txt file:

Sitemap: <sitemap_location>

The <sitemap_location> should be the complete URL to the Sitemap, such as: http://www.example.com/sitemap.xml

This directive is independent of the user-agent line, so it doesn't matter where you place it in your file. If you have a Sitemap index file, you can include the location of just that file. You don't need to list each individual Sitemap listed in the index file.

You can get all the facts about this and everything you need to know about sitemaps at sitemaps.org.

You May Also Like

7 Responses to “Sitemap autodiscovery for the Big 4 via robots.txt”

  1. drum says:

    Are you at the conference today?

  2. DazzlinDonna says:

    No, I couldn't make it to the conference.

  3. drum says:

    Are you going to SMX in June? I'll be there.

  4. DazzlinDonna says:

    Probably not, but if I do, I'll let ya know.

  5. pittbug says:

    Here's the direct URL to the FAQ which describes the implementation: http://www.sitemaps.org/protocol.html#informing

  6. simleon says:

    Dazzlin, never in Europe? I want to have the opportunity to meet the best SEO girl in the world….
    :-) Thanks for this wonderfull blog.

  7. DazzlinDonna says:

    Simleon, I'd never rule Europe out completely, but it's not very likely. Well, at least not JUST for business reasons. If I manage to get there just for fun, I could throw business in as well. I'll be sure to let you know if it ever happens. :)