New sitemap standards make it easier for all search engines to find your site

Sitemaps have been about for a while. They were originally created a visual guide to your website. As websites became larger and more complex webmasters used them as a way to view the entire structure of a website to find the connections between pages. Think of them as a kind of interactive table of contents and index rolled into one.

In recent years the search engines have been viewing site maps as a good way to index websites and find all the related content. Google led the charge to make this a mainstream web tool when it introduced XML sitemaps. Creating and submitting an XML sitemap to Google is still the best way to get your website completely crawled by Google. It has become a standard SEO best practice.

As with many things on the web, Google, Yahoo, AOL, MSN and Ask all had their own specific way to use and submit sitemaps in the format they dictated. About six months ago a couple of these folks joined forces to create a standardized method of sitemap creation. They formed an organization called Sitemaps.org and agreed on using the XML sitemap format.

Sorry for the history lesson, but this is really good news for small business web site owners because it will greatly simplify the process of creating and submitting your sitemap and in doing so enhance your chances of getting your entire website indexed by all the search engines.

So, in today's article I'm going to tell you how to create and submit your XML sitemap.

What is a sitemap?

According to Sitemaps.org: "Sitemaps are an easy way for webmasters to inform search engines about pages on their sites that are available for crawling. In its simplest form, a Sitemap is an XML file that lists URLs for a site along with additional metadata about each URL (when it was last updated, how often it usually changes, and how important it is, relative to other URLs in the site) so that search engines can more intelligently crawl the site."

Building an XML sitemap

While there are several ways to actually create an XML sitemap I like XML-Sitemaps.com The good news is that if you have a rather small site (under 500 pages) you can simply use their free tool to create an xml sitemap. Over 500 pages and you need to get the $19.95 download.

Getting Your Sitemap found

Once you create your sitemap it's still a good idea to create a free Google Webmaster account so that you can submit your sitemap directly to Google Sitemaps

Robots.txt file

Now here's where the coming together of the search engines gets really nice. Not only will Ask, Google, Yahoo, AOL and MS Live accept the XML format for your sitemap they have also decided to accept an easy, auto-discovery method. In other words you won't have to figure out how to submit to each as they will find your sitemap if you direct them using your robots.txt file. A robots.txt file is a very simple file that resides on your server giving information for the search engines. Many use this file to tell the search engines not to find certain information.

If you already have a robots.txt file, you can simply add the line of code below anywhere to it. If you don't have one, simply create a file in notepad or other text editor, add the code below, save it as robots.txt and upload to the root of your site. (Obviously you need to put the actual URL of your site in here)

Sitemap: http://www.yourwebsite.com/sitemap.xml

Advanced tip - If you use the paid version of xml-sitemap generator you can set up what's known as a cron job (it's a Unix only thing) and have your site crawled on a weekly basis and update your sitemap. This is really great if you blog several times a weeks on your domain as it adds your newest posts. Ask your web host about this one.

Useful links
Sitemaps.org http://www.sitemaps.org
Google Webmaster Blog - http://googlewebmastercentral.blogspot.com/