XML Sitemaps are an easy way for webmasters to let search engines know which pages on their site their robots can crawl.

A typical XML sitemap file lists each URL, along with information about when it was last updated, how often it normally changes, and how important it is relative to other pages on the site. This helps search engines crawl your site more intelligently.

In November 2006, Google, Yahoo! and MSN have joined forces to support a new industry standard for sitemaps: Sitemaps 0.90. As long as webmasters follow the protocol, they can ensure that their sites are fully and consistently indexed in all major search engines (a real step forward). This article is important for all those who have pages missing or misclassified.

The joint venture’s official site is at sitemaps.org and contains a lot of information about the new standard and its syntax. What the site singularly fails to do is properly explain how to submit your sitemap to the big three! The suggested format on the site:

currently not working on any of the three sites! Until you do, this short article provides instructions on how to (a) create your sitemap and (b) how to submit it to each of the three major search engines…

Creating your sitemap

Some hosting providers (for example, 1and1) provide utilities through their web control panel to create your sitemap, so always check with your provider first. If this service is not available, visit xml-sitemaps.com and enter your site’s URL in the generator box. Copy and paste the resulting sitemap into notepad, then save and upload it to your site with the filename: sitemap.xml

If you want to validate the XML before uploading it to search engines (useful if you’ve made manual edits), check out the XML validator (on the same site) where you can enter your sitemap URL and compare it to the standard.

Send sitemap to MSN

MSN has yet to implement a formal interface for Sitemap submission (as of July 2007). To monitor the situation, visit (from time to time) the official MSN Livesearch blog (where future announcements are likely to be found).

While MSN has yet to implement a front door, there is a recognized back door to submitting your sitemap to the MSN search index; i.e. addition.com! You must use the following syntax directly in your browser’s URL box:

Since February 2005, moreover.com has been the official RSS feed provider for the myMSN portal (see press release) and reliable evidence suggests that submission to Plus will result in MSN crawling your pages within 2-3 weeks. .

Note that while MSN doesn’t yet support drop shipping, they suggest on their blog that you add a reference to your Sitemap in your robots.txt file (something that sitemaps.org now supports). For example:

This would tell MSN (and all other engines) to crawl your sitemap file, but not your cgi-bin directory. For more information on how to deploy a robots.txt file (at the root of your site’s web server), visit: http://www.robotstxt.org

Submit sitemap to Google

Google originally developed the XML schema for sitemaps and has developed a dedicated webmaster portal, from where you can submit your sitemap:

First, you need to tell Google all the sites you own, and then verify that you actually own them. Verification is achieved by adding a meta tag between the main tags on the home page of your site. The tag syntax is as follows:

There are full instructions on how to do this on the Google site.

Submit sitemap to Yahoo

Yahoo follows a similar approach to Google. Again, there is a dedicated service for webmasters (Yahoo! Site Explorer) and a procedure to verify your ownership of the site. First go to:

Add a site, then click the check button. You can then download a verification key html file, which you will need to upload to the root directory of your web server. You can then go back to Site Explorer and tell Yahoo to start authentication. This will take up to 24 hours. At the same time, you can also add your sitemap by clicking the manage button and then adding the sitemap as a source.

Send site map to Inquire

Ask follows a simpler approach than the other three. To submit your sitemap, simply enter a ping URL, followed by the full URL where your sitemap is located:

After clicking return, you will receive a reassuring message from Ask that they have received your submission. Very neat!