Sitemaps have become an SEO tool these days. Search engines cannot reach many kinds of pages easily. The unreachable pages include dynamically generated pages and FLASH and AJAX pages.

Discussion forums and e-commerce (database) sites are examples of sites with difficult to index pages. These kinds of sites would typically have thousands of pages, most of which would not be indexed by search engines in the normal course.

It is here that sitemaps come to the rescue of the Webmaster.

What Are Sitemaps?

In earlier days, sitemaps were simple. You just created a list of links to all pages of your site and converted that list into an HTML file. Then you uploaded that file, probably named sitemap.html, to your site.

These sitemaps could be meant mainly for visitors or for search engines or for both.

Sitemaps Get Formalized

Sitemaps these days can be XML, ROR or text formatted files. While the text sitemaps are just a list of URLs, the XML and ROR sitemaps provide additional details about each of the URLs.

In the XML sitemap, you provide information such as the priority of each URL relative to other URLs, the typical frequency of changes to a URL and the last date of its modification. ROR sitemaps allow even more information to be included.

What's Special About New Sitemaps?

An XML sitemap helps the search engines to crawl a site more intelligently. The engines can give greater weight to the more important pages, and can also schedule the crawling to synchronize with the page change frequency. Search engine crawling can thus cater better to Webmaster preferences.

Major search engines like Google, Yahoo, MSN and Ask have adopted the XML sitemap protocol. Search engines would also recognize a line in robots.txt file that tells them where they can find the sitemap:

Sitemap: http://www.yourdomain.com/sitemap.xml

Submitting Sitemaps Direct to Engines

Google and Yahoo provide facilities that enable you to submit your sitemap direct to them. You log into your Google or Yahoo account and follow the instructions for submitting the sitemaps.

The advantage of this option is that Google and Yahoo provide reports to you about your site. Indexing status, traffic statistics and keywords used to land on your pages are some of the information you could get in this way.

To start a direct sitemap submission process, do the following:

1. Upload your XML sitemap to the root directory of your website

2. Go to the relevant page indicated below:

Google: https://www.google.com/webmasters/tools/

Yahoo: https://siteexplorer.search.yahoo.com/submit/

3. Login to your Google or Yahoo account, as applicable

4. Now follow the instructions for submitting your sitemap. Both engines require you to complete a site verification or authentication procedure to check that you are the Webmaster or owner of the concerned site.

5. Once the verification or authentication is complete, they will arrange to crawl your site. Indexing might take time, however.

6. The major benefit is that you would now have access to considerable information about your site.

Don't Forget to Keep the Sitemap Updated

Creating and uploading a sitemap is not enough. You must keep it updated as more pages are added (or pages are deleted). There are sitemap generators to help you. Just search for them in Google.