Is googlebot the slowest bot?
-
This morning, I wrote a breaking news story about a "Wolf of Wall Street"
It was published at 12:05:49
Googlebot, which used to be on my site within a minute or less, didn't bother to visit for 53 minutes. And now, 32 minutes later, even though it has been crawled, this story doesn't even show up in google search.
Except that it is in the top 10 stories today, at #2, so the headline appears in every page on the site, so every page that has been crawled today, around 10 minutes after it was published, contains that text, so they show up. EINnews, which also crawls our pages is listed for the headline text.
Finally, the page turns up in search results 4 hours later, and says that it is 4 hours old.
Does anyone else see this slow motion mode?
If you do see this, what is wrong with the site that causes this recalcitrant behavior?
The headline of the story is "A 'Wolf of Wall Street' Raided By FBI In Florida"
and the link is http://shar.es/1bW5Sw
-
Hi Alan,
Are you still pushing Google News tagged XML sitemaps when publishing articles as well? Looking at the ones currently on your site I don't see any new ones referenced since October 2014. And it looks like there's a lot of current mapping that could be updated.In general the site seems a little low in the loop of the major news cycle and would have a lower crawl/index priority on big stories behind the CNNs, Foxs, and Yahoos of the world.
It also doesn't seem to be in the Google News index: https://encrypted.google.com/search?hl=en&q=site%3Anewsblaze.com#hl=en&tbm=nws&q=wolf+of+wall+street+site:newsblaze.com
Google's Guideline to Google News Publisher inclusion is straightforward, and fairly thorough. https://support.google.com/news/publisher/answer/40787?hl=en If you get included via those means, you should see your news articles appearing very quickly.
-
Thank you for looking Ryan.
Google ignores out news map because they removed us from new for a reason they didn't disclose, about 2 years ago.
So I haven't been checking on that map.
I've been trying to find time to change over from my custom-built CMS to Wordpress, and thought I'd reapply after I did that, but I'm 6 months behind my schedule to get that done. (had problems with the page design and the data conversion)
Yes, we're much smaller than the others, but 4hours for a page to show up in the index must mean something else is going on, and I can't work out what that could be.
I'll see if I can get my redesign back on track, and that will make the site more mobile-friendly.
Have you seen anything like that 4 hour gap before? I will track the next few stories I publish too, and report back.
-
Your gap will disappear if you get back into the News index. Best of luck!