How do can the crawler not access my robots.txt file but have 0 crawler issues?

[will_l]

So I'm getting this errorOur crawler was not able to access the robots.txt file on your site. This often occurs because of a server error from the robots.txt. Although this may have been caused by a temporary outage, we recommend making sure your robots.txt file is accessible and that your network and server are working correctly. Typically errors like this should be investigated and fixed by the site webmaster.https://www.evernote.com/l/ADOmJ5AG3A1OPZZ2wr_ETiU2dDrejywnZ8kHowever, Moz is saying I have 0 Crawler issues. Have I hit an edge case? What can I do to rectify this situation? I'm looking at my robots.txt file here: http://www.dateideas.net/robots.txt however, I don't see anything that woudl specifically get in the way.I'm trying to build a helpful resource from this domain, and getting zero organic traffic, and I have a sinking suspicion this might be the main culprit.I appreciate your help!Thanks!

[badgergravling]

Well technically if it can't crawl your site, it won't be able to find any issues...

I think this may be an error with the Moz crawler rather than any crawlers - checking with site:dateideas.net pulls up some results, although they seem to be tag and archive pages at the moment, which I'd be looking to noindex to avoid duplicate results in the future... You can always check Google's crawling by making sure you've registered your site with Google Search Console, and I'd suggest also making sure you've submitted your sitemap.

I have had issues in the past with the Moz bot struggling to crawl on specific hosts, so it might be that... The new crawl seems to work fine for me at the moment, but is it possible your site was down at the moment Moz was crawling it? You can always request a new crawl and monitor your site uptime to see if there's an issue...

[Roman-Delcarmen]

This just an advice that worked for me in the past for the same issue and of course is not in the user guide. Simple a delete the website or project from Moz Pro and create a new one for the same domain, I dont know why but all the errors and Issues disappeared.

[tawnycase]

Hey there! Tawny from Moz's Help Team here.

It looks like your site's robots.txt file is returning some errors to our tools. When I try to visit the robots.txt file from the root domain, which is where our crawler starts, I get a warning that the DNS address can't be found: https://www.screencast.com/t/ezfsiyVso4B9 That same file is returning a 503 error to our crawler: https://www.screencast.com/t/ROlNo8AQz

That robots.txt file doesn't redirect anywhere, so you may want to consider putting in a redirect there to your robots.txt file at http://www.dateideas.net/robots.txt.

The reason you're seeing 0 issues reported is that we weren't able to reach your robots.txt file, so we stopped crawling and didn't have any issues to report.

I would speak to your web developer or whoever manages your site for you about making sure that your robots.txt file is fully accessible to our crawlers and can be reached in a browser.

I hope this helps! If you've still got questions, feel free to shoot us a note at help@moz.com and we'll do our best to sort everything out with you!