How To Prevent Crawling Shopping Carts, Wishlists, Login Pages

[Romancing]

What's the best way to prevent engines from crawling your websites shopping cart, wishlist, log in pags, ect...

Obviously have it in robots.txt but is their any other form of action that should be done?

[ShaMenz]

Hi Shawn,

It is inadvisable to use more than one method of blocking bots from specific pages on your site.

There is a very good explanation of this in this blog post from Lindsay - Robot Access and Indexation Restriction Techniques: Avoiding Conflicts. Definitely worth reading!

Hope that helps,

Sha