Robots.txt error message in Google Webmaster from a later date than the page was cached, how is that?
-
I have error messages in Google Webmaster that state that Googlebot encountered errors while attempting to access the robots.txt. The last date that this was reported was on December 25, 2012 (Merry Christmas), but the last cache date was November 16, 2012 (http://webcache.googleusercontent.com/search?q=cache%3Awww.etundra.com/robots.txt&ie=utf-8&oe=utf-8&aq=t&rls=org.mozilla:en-US:official&client=firefox-a).
How could I get this error if the page hasn't been cached since November 16, 2012?
-
I've noticed several discrepancies in google's cache system. It seems many of their documents lag and are not updated immediately. It could just be errors in cross data population. If you really want to know the last time the Google bot visited your website then you will want to visit your server logs.
If your server logs don't show a visit from google on the 25th then we really do have to wonder. My guess is that the webmaster tools is reflective of the correct date. Either way, I'd check for the error they are reporting.
-
That's what our next move is. I'll let you all know what comes of it. Thanks for the response!