How can I solive this ?
-
-
Hi there,
I've tried to access your robots.txt file and I'm getting a 502 error.
https://www.screencast.com/t/QP3b08E86qP
You can use this third party tool to see this response http://httpstatus.io/
If we can't crawl your robots.txt file we can't crawl your site.
I would recommend reaching out to your website administrator or host to find out how you can solve this error.
If you're still stuck please reach out to the help team help@moz.com.
Thanks!
Jo
-
Hello Jo,
Here's my robots.txt file : https://www.hero-t3ch.com/robots.txt
-
It looks like there is still a connection timeout for the non-www version of your site and robots.txt file
I've run a test using this tool again
And this is what I'm seeing:
https://www.screencast.com/t/ll7WdleY9f
Our crawler always starts with the non-www version of your site and crawls from there.
To fix this I would recommend speaking to your website administrator or host about correctly redirecting your non-www pages so we can crawl your site.
I hope this helps!
Jo