How To Sitemap: Creating XML Sitemaps

by Stephanie Woods October 21st, 2009 

Hiya Google spiders,

While you're here, can you please make sure you check out our main product pages since we make changes to them frequently? Also, we added some new blog pages, so be sure to check those out as well.

Have a great week!

Thanks,

Steph

Wouldn't it be great to leave a note providing suggestions to search engine spiders about how you want your site crawled when they visit your site? Well guess what? You can.

Spider dog. Courtesy of epicfunnypictures.com.

Spider dog. Courtesy of epicfunnypictures.com.

A XML sitemap is similar to the notes left behind by my well organized sister-in-law when I dogsit for her and my brother. Her notes let me know when to feed the dogs, how often and where to find everything.

A note written for the search engines spiders provides similar details about how to properly crawl your site.

Back in 2005, Google developed a new sitemap protocol in response to the fact that sites kept getting bigger and more complex (foreward ho). The XML sitemap is born. It is designed to make good use of the search engine spiders' time by enabling the engines to index pages from your site more efficiently.

XML sitemaps do not guarantee that all of your pages will get indexed, but they do help. XML sitemaps also provide you with the opportunity to let search engines know what pages you feel are the most important (i.e. main product or service pages) the last time the content of each page was modified, and how often you expect that content on each page will be updated.

Is Sitemap Submission Outdated?

There are people on both sides of the fence when it comes to the practicality of submitting sitemaps to search engines. One of the reasons why some people are skeptical is because your site will most likely be found by spiders regardless.

In my personal experience the pros outweigh the cons.

As with many SEO tactics, I am a strong believer that every little bit helps. In most cases it won't have a negative effect to submit your XML sitemap to search engines. So why not?

First of all, it ensures the search engines index your pages. Secondly, you can submit your XML sitemap to one (or all) of the available (and free) webmaster tools provided by the major search engines that help point out crawl errors.

XML Sitemap Creation 101

It is possible to create a sitemap from scratch; however, many people use a sitemap generator and then tweak it. Let's face it, it's a tedious task to manually create a sitemap and an automated sitemap generator works.

If you have a static site, try out this XML Sitemap generator.

If you utilize a Content Management System (CMS), there are various plugins that you can make use of for sitemap submissions. Some recommended plugins for CMS' are detailed below.

Once you've created your XML sitemap you can manually play around with the various XML tags and attributes that can be used in a sitemap file, also known as XML schema.

Not quite this manual. Photo courtesy of umanitoba.ca.

Not quite this manual. Photo courtesy of umanitoba.ca.

The following are the XML schema available and how you can use them for each URL on your website.

  • <lastmod>
    • Lets spiders know when you last modified the page.
  • <changefreq>
    • Provides information on how frequently the page is likely to change. This is more of a hint though. It doesn't mean the engines will follow that command.
  • <priority>
    • This tag enables you to tell spiders which pages you feel are most important. Main product or service pages are generally given a higher priority. Labeling all pages as high priority will not help you because priorities are relative to other pages on your website.

XML Sitemap Plugins for CMS

If you have a WordPress site you can try this Google XML Sitemap Generator plugin. It was designed specifically for Google, but it works with the other major search engines as well. A good benefit of this plugin is that every time an update is made to the site, like a new blog post, or a new product is added, the module submits the changes automatically. This can help new pages get indexed more quickly (but not always guaranteed).

For Drupal sites try the XML Sitemap Module. Installing this module makes XML sitemap submission to the engines a breeze. Like the WordPress plugin, this module submits any changes made to the site automatically to the search engines.

I haven't had the chance to personally work with a Joomla CMS yet and therefore am not sure which plugin is the best. On the Joomla site this Xmap plugin seems to be the most popular and has the highest number of votes.

If you're using any other CMS, a simple Google search for XML sitemap plus the name of your Content Management System (e.g XML sitemap Joomla) should provide you with some worthy XML sitemap plugin information.

XML Sitemap. Check. Now What?

If you have a static site (as opposed to a CMS as discussed above) the first thing you need to do is upload the XML file to the root of your domain (add the URL to your site). This will require logging into the FTP with your credentials. The most commonly used format is this: www.yoursite.com/sitemap.xml.

Image courtesy of seraphicpress.com

Image courtesy of seraphicpress.com

Once you have uploaded the XML file to your domain, you can edit your robots.txt file to include the sitemap URL. A robots.txt file tells search engine spiders what they shouldnt crawl. A sitemap tells them which pages they should crawl. Together they compose a detailed note of recommendations you can leave for search engine spiders.

Spiders are programmed to look for the robots.txt file when they crawl your site. As of 2005 you can include a line in the robots.txt to tell the spiders to look for your XML sitemap. If you have multiple sitemaps (for large sites) you can include more than one sitemap per robots.txt file.

You can direct spiders to your XML sitemap by placing this line anywhere in the robot.txt file:Sitemap: http://www.example.com/sitemap.xml

Not having a XML sitemap isn't the end of the world. In fact, your site can still do extremely well without one.

It is, however, one of the only ways to directly communicate with spiders and leave them a little to-do list. Enticing isn't it?

My next post will detail how to submit your sitemaps to the major search engines.

——————————————————————————————————

Stephanie Woods is an internet marketer living in Kelowna, BC. You can find her at her (very) new marketing blog . She has also been known to frequent Twitter if you're into that sort of thing.

Stephanie Woods

Stephanie Woods is a Search Specialist at Blast Radius in Vancouver, BC.

Steph Woods SEO

You May Also Like

12 Responses to “How To Sitemap: Creating XML Sitemaps”

  1. Hi!

    I did a sort of similar article, but comparing XML sitemaps vs. regular HTML site maps…apparently Google slightly prefers HTML sitemaps.

    If you're interested, the article is here http://www.seofool.com/xml-sitemap-vs-html-sitemap-which-is-better-for-search-engine-optimization

  2. [...] How To Sitemap: Creating XML Sitemaps [Learn SEO], The SEO Scoop [...]

  3. Alex says:

    Sitemap helps SE to crawl/ index your site, but it ialso helps ou to see how you do in this or that SE (oages in the index stats.

  4. Sarah says:

    Hi Stephanie – Do you know how to create an XML sitemap for a website with a Flash navigation?

    I keep thinking there is a magic bullet somewhere, other than having to manually compile a list of pages…ugh. Xenu's Link Sleuth, XML sitemap generators – none of 'em work (darn flash nav).

  5. Stephanie Woods says:

    John – I saw that Webmaster video by Matt Cutts as well and was also surprised to hear him say if you're going to do one over the other, HTML is better. But as you mentioned in your article, you might as well do both! I have seen direct results from submitting XML sitemaps in the past.

  6. Hi Stephanie,

    Speaking of…did you see the video yesterday where they shaved Matt Cutts head? Kinda funny I guess *g*

    John Elder
    seoFool.com

  7. Stephanie Woods says:

    John – No I didn’t see the video with Matt Cutts Shaving his head, but I did notice some a couple tweets about it yesterday. When I get a chance I’ll check it out!

    Sarah – Good question! I haven’t had any experience working with sites that have Flash nav as I am generally involved with a website right from the get-go (and wouldn’t use Flash nav to begin with). If possible, I would recommend recoding the nav using a CSS menu. The reason being is that while Google can index the text on the menu, it can’t follow the links in drop down menus – which makes it tough for those pages to get indexed. Sorry, I know I didn’t answer your question but I don’t actually know what the answer is!

    Alex – You sound like you’re making a good point, but I don’t quite understand what you’re trying to say. Feel free to clarify. =)

  8. I don't think it is outdated but it is one of those things that surely helps out the over all goal a website is trying to accomplish.

  9. Kris Malena says:

    I have always been a little baffeled by this sitemap creation. This article helps me understand the importance of them a little more.

  10. Teasastips says:

    That's why I love WordPress! Before I used Blogger and couldn't figure out why my posts were not getting picked up. Just a few weeks with WordPress using the Google XML Sitemap plugin and it has made a huge difference. Great post!

  11. [...] you are interested in learning how to create one, be sure to check out the precursor to this post: How To Sitemap: Creating XML Sitemaps. Monty & Spider by [...]

  12. Andre Colt says:

    I do both html and xml sitemap.Html sitemap to for the users while xml sitemap to easy crawl my site pages by the SE.I think xml sitemap is important nowadays because when I do an html sitemap and check if my site is index my pages did not appear and when I build an xml sitemap using xml sitemap generator luckily it appears.