Website can't be crawled

[Yenlo]

Hi there,

One of our website can't be crawled. We did get the error emails from you (Moz) but we can't find the solution. Can you please help me?

Thanks,

Tamara

[jocameron]

Hi there,

Jo here from the Moz Help Team.

The best way to investigate issues like this, is to start with our troubleshooting guide here https://moz.com/help/moz-pro/site-crawl/crawl-troubleshooting

I have tried this myself and I've gotten to step 3 where I can see that there is an error when I try you site in this third party status checker.

This is an indication that crawlers like ours and this tools are being blocked from accessing your site at a server level.

Please check with your website admin to see if they are not blocking AWS, or rogerbot. Once you've fixed this with your website admin you can trigger a recrawl of your site if you have a Medium subscription or higher https://moz.com/help/moz-pro/site-crawl/overview

If you're still stuck please reach out to help@moz.com.

Best

Jo

[Yenlo]

Hi Jo,

Thanks for checking! I will ask our website provider (HubSpot) if they block AWS or rogerbot.

Thanks,
Tamara