Have a Robots.txt Issue

[primemediaconsultants]

I have a robots.txt file error that is causing me loads of headaches and is making my website fall off the SE grid. on MOZ and other sites its saying that I blocked all websites from finding it. Could it be as simple as I created a new website and forgot to re-create a robots.txt file for the new site or it was trying to find the old one? I just created a new one.

Google's website still shows in the search console that there are severe health issues found in the property and that it is the robots.txt is blocking important pages. Does this take time to refresh? Is there something I'm missing that someone here in the MOZ community could help me with?

[OlegKorneitchouk]

What does your robots.txt file contain? (or share the link)

Try removing it, clearing server cache and fetching as google again.

[primemediaconsultants]

This seems very helpful as I did remove it, and fetch as google, but i'm a complete novice. How do you clear server cache?

[Martijn_Scheijbeler]

You not always have to do this, if you would go to domain.com/robots.txt then it should be removed maybe already. If that's the case you should be starting to see an increase in the number of pages crawled in Google Search Console.

[MattRoney]

Hi primemediaconsultants!

Did this get cleared up?