Welcome to the Q&A Forum

Possible Solitions for your problem: .htaccess authentication blocking robots.txt http://www.irishwebmasterforum.com/server-technical-administration-tips-and-queries/10892-htaccess-authentication-blocking-robots-txt.html 301 redirect. How to make an exception for the robots.txt http://forum.cs-cart.com/topic/23747-301-redirect-how-to-make-an-exception-for-the-robotstxt/ 1. Canonical robots.txt http://digwp.com/2011/03/htaccess-wordpress-seo-security/ General .htaccess tutorials: http://httpd.apache.org/docs/2.0/howto/htaccess.htmlhttp://httpd.apache.org/docs/2.0/misc/rewriteguide.html