Cannot Crawl ... 612 : Page banned by error response for robots.txt.

[1sixty8]

I tried to crawl www.cartronix.com and I get this error:

612 : Page banned by error response for robots.txt.

I have a robots.txt file and it does not appear to be blocking anything

www.cartronix.com/robots.txt

Also, Search Console is showing "allowed" in the robots.txt test...

I've crawled many of our other sites that are similarly set up without issue.

What could the problem be?

[DirkC]

Your robots.txt renders in a browser - but from technical perspective it generates a 403: Forbidden (check http://www.cartronix.com/robots.txt with web-sniffer.net)

Moz will not crawl if your robots.txt is returning a 403 (see answer from Chiaryn Miranda / Moz on https://moz.com/community/q/without-robots-txt-no-crawling

Quote: "The only commands from the http responses that we consider to block our crawler from accessing a site would be a 403: Forbidden error or a 5xx error."

Dirk

[1sixty8]

Thank you for the responses.  Can you give me any direction on how to correct this?  I am lost

[1sixty8]

I think I figured it out... For some reason, robots.txt was set at 600...I changed it to 644... I will run crawl again... Thanks.

[DirkC]

Great - just checked the robots.txt with web-sniffer & shows a 200 status now so crawl shouldn't be an issue.

Dirk

[1sixty8]

Thank you everyone... I'm learning!  And you are helping!