Site Crawl 1-page 301 status error but httpstatus.io says its 403
-
I am trying to run a site crawl for my website and MOZ is only resulting in 1 page crawled with the home page URL Status Code of 301.
However when I run it in httpstatus.io it is giving me a 403 status error.
Im curious as to why MOZ is saying its a 301 and httpstatus.io is saying 403. Is there anything I can do in MOZ first to get the site crawled before asking my developers to look into the 403 error?
-
Hi John - Thanks for posting
Moz is starting at HTTP, so if you check httpstatus.io from HTTP, you will see a 301 to a 403 response. You would want to have your developer check to see if they are blocking our user-agent "rogerbot" or any AWS IP's because it doesn't look like we can crawl past the root domain.
If you need further assistance, feel free to reach out to help@moz.com.
-
Would this also block the Googlebot?