Telling search engines where your sitemap is...

Those of you interested in SEO really need to get to grips with the sitemap:url line in your robots file... The what? I hear you cry... Well let me explain.

For a long time now there has been an accepted protocol for sitemaps that many of the top search engines have adopted (check out: http://www.sitemaps.org/ for more info) but it's always been a bit of a headache going to each of the big search engines and telling them about your sitemap so that robots can index it right?

Well Google introduced the Webmaster Tools dialogue so that you could tell Google where your sitemap was and use it's tools to evaluate any crawl errors (404s etc). Well there is a really really simple way to tell all of the "big four" (namely Microsoft, Yahoo, Google and Ask).

All you need to do is add one line to your robots.txt file and those search engines will find it. So what's the line?

Simply:

sitemap:<url>

Where <url> is the URL of your sitemap file. In our case:

sitemap:http://www.coderequired.com/sitemap.xml

Still not sure? Check out sitemap.org's Informing search engine crawlers article

Accessibility and anti-robot testing

One thing that PAS78 hopefully will bring to more people's attention is the use of graphics for anti-robot testing... Also known as the obscured numbers in a graphic that you have to type into a box to confirm your registration or if your a complete marketing-speil word-monkey then 'Captcha' or 'Turing testing'.
PAS78 raises the issue, quite rightly, that if a user has any type of sight impairment the use of this technology could in fact disable the use of these forms unless there is an alternative method available.

Hotmail does include an audio version of it's Captcha image, so that a user can listen to the numbers being read, and then enter them into the verfication form field (although if you have ever attempted to listen to one of these it's incredibly diffcult to inteprete what the hell is being said as they have distorted the sound to deter robots with sound analysis!)

Google has gone to the extent of using SMS to send a user a passcode, which is a novel idea, but SMS isn't completely accessible yet either so that isn't a solid solution - and let's face it - not exactly cost effective either if you have to set up an SMS gateway etc etc.

Unfortunately at the moment there is no surefire ideas on how to implement accessible Captcha methods... Virtually all methods can be hacked and using these methods of human detecting inevitably will be difficult to use for some users... A 'code' based system such as those mentioned above may be difficult for dyslexic users; The common 'distorted image' method is difficult for those with visual impairments; and so on and so on...

So what is the solution? Well I'm not about to answer that here but do make sure you consider these users next time you implement some form of anti-robot testing in your next application... Create an audio version if you are going to use the recognised 'distorted image' method, try to avoid complex logic and always consider ALL of your users!

related link