Quick robots.txt check

[neooptic]

We're working on an SEO update for http://www.gear-zone.co.uk at the moment, and I was wondering if someone could take a quick look at the new robots file (http://gearzone.affinitynewmedia.com/robots.txt) to make sure we haven't missed anything?

Thanks

[Benj25]

It's ok but very basic:

User-agent: *

Disallow: /myaccount/

Sitemap: /sitemap.xml

Do you want to stop crawlers from accessing the login page for example?

Ours is something like this:

# Disallow All Engines From Admin and Login

User-Agent: *
Disallow: /index.php/

User-Agent: *
Disallow: /index.php/admin/

User-Agent: *
Disallow: /customer/account/login/

# Sitemap Files

sitemap: http://www.worldofbooks.com/sitemap.xml

[Benj25]

Plus - look around! Check out other companies Robots.txt file

http://edition.cnn.com/robots.txt

http://www.nytimes.com/robots.txt

You can see what they do not think is relevant for search engines to be looking at.