Questions
-
404 Crawl Error on Homepage
Hi there! I just ran the site through our crawler using a few different cURLs and it does look like the homepage URL is returning a 404 to our crawler: http://www.screencast.com/t/fgoeCGHu and http://www.screencast.com/t/SRzcBXJIZmw. It may be a user agent specific issue, where the server may be trying to specifically block our crawler, but it is set to use the wrong http status so it returns a 404 rather than a 403. If you look at your server logs for the time of the crawl, you should be able to see the exact response of your server to our crawler, rogerbot. I would recommend working with your webmaster to look into how the site is responding to our crawler further. If you have more questions or need specific information, I would recommend emailing help@moz.com so that we can look into the issue further for you.
Other Questions | | ChiarynMiranda0