Mozbot IP / reverse DNS

[RobertN-London]

Hi,

due to security measures (which i imagine a lot of enterprise clients are starting to use now) we're very restrictive of automated traffic coming to our websites... however we're also blocking the mozbot.

Will Moz have a fixed IP address/range or reverseDNS look up approach that can be used in the future?

thanks!

[lauren.s]

Hello!

For our link crawler, you will want to make sure you are blocking "dotbot" specifically.

Unfortunately we do not use a static IP address or range of IP addresses or plan to do so in the future, as we have designed our crawler to have a dynamic approach. This means we use thousands of dynamic IP addresses which will change each time we run a crawl. This approach gives us the best dynamic view of the web, but it can make us incompatible with some servers or hosting providers.

The best option I'd be able to suggest would be to identify our crawler by User-agent: dotbot

I apologize for the inconvenience this causes!

[RobertN-London]

thanks! Will you allow for reversDNS lookup?

similar to: https://support.google.com/webmasters/answer/80553?hl=en