Questions
-
What is the IP address of the MOZ crawlbot?
Hi, Moz is hosted on Amazon. See this sample log entry from today: 54.146.235.41 - - [17/Sep/2017:09:48:13 +0200] "GET /best-deals.html/page/12/ HTTP/1.0" 200 15803 "-" "rogerbot/1.2 (https://moz.com/help/guides/moz-procedures/what-is-rogerbot, rogerbot-crawler+aardwolf-production-crawler-97@moz.com)"[XF 54.146.235.41] That IP is part of Amazon's ASN. whois -h whois.radb.net -- '-i origin AS16509' | grep ^route You can either use something like mod_authz to allow the domain amazonaws.com to connect or you can get each IP from AS16509 and allow that. Hope it helps. Regards Chris
Getting Started | | chrislemmer0