How do I complete a reverse DNS check when completing log file analysis?

[daniel-brooks]

I'm doing some log file analysis and need to run a reverse DNS check to ensure that I'm analysing logs from Google and not any imposters. Is there a command I can use in terminal to do this?

If not, whats the best way to verify Googlebot?

Thanks

[TylerReardon]

Hey Daniel,

If you want to verify that a user-agent is actually Googlebot, you'll want to use a log file analysis tool to aggregate all of the IP addresses associated with Googlebot. Once you have a list of IP addresses, you can perform a reverse DNS lookup to verify whether the IP addresses are actually associated with Googlebot or not.

If you're on windows/pc these steps should work:
https://www.serverintellect.com/support/dns/reverse-dns/

If you're on mac try these steps:
1. open Terminal
2. type "host" + ip address
 for example: "host 66.249.66.1"
3. hit enter
4. view results. For example: "1.66.249.66.in-addr.arpa domain name pointer crawl-66-249-66-1.googlebot.com"

If the results are from Google.com or Googlebot.com, you can be sure it's actually Google crawling your site. Unfortunately, I don't know of any faster ways to achieve these results. I'm sure there's a tool out there, I just haven't found it yet.

This might also be a good resource for you: https://support.google.com/webmasters/answer/80553?hl=en

Good luck!
-Tyler

[daniel-brooks]

Hi Tyler,

Thanks for your reply. I managed to get down to 98 unique IPs and ran a bulk reverse DNS/IP Look-up using this tool:

https://www.infobyip.com/ipbulklookup.php

Thanks for your help though!

[TylerReardon]

That's awesome! Glad to know there's a bulk tool out there!