What does the blocking_google property in the CSV export mean, and what to do about it?
-
The CSV export for Crawl Diagnostics contains a column named "blocked_google". It states a blocking date/time but doesn't occur on all our webpages, not even on all pages of the same type / structure.
There are no other flags on these records that would explain a blocking of Google; all other agents are not flagged, and our robots.txt doesn't contain any blocks either. The only flag the records have in common is "Page Title > 70 characters". Of course, I could just assume this is the reason for the "blocking_google", but is it?
What evaluation makes the crawler fill in this property, and how to handle/solve it's occurrence?
-
check that those pages don't have some like:
meta name="robots" content="NOINDEX,NOFOLLOW" />
in the source code
-
Thank you for your response.
Our CMS generates the following meta-tags on the affected pages:
Pragma is for cache control, content-type and google verification are clear. P: is Pinterest Social media, the rest is standard again. I searched the source of the page for "robots" as well, but no results.
-
can you give an example of one of the pages the is "blocked_google"
Btw try screeming frog ( you can do 500 pages for free), it should pick up the same error and might explain why.
-
Thank you again for your response.
The following link forwards to one of the affected pages: http://bit.ly/1b3IVYB
I'm downloading Screeming Frog right now and give it a try, thank you for the advice.
-
that page seem fine, and it also indexed by google, so I'm not sure whats the story with that. Might be best to contact seomoz support (help@moz.com I think)