Google fetch (more problems)

[WallerD]

This post is deleted!

[Hutch42]

It looks like a lot of those blocked pieces are not on your site but are located on other sites and you are pulling them onto yours.  Now it also looks like, besides the gallery, the rest of these pieces really are not important (social media buttons, placeholder images) so you should not be worried about them. The only one you should look into more is the Gallery box script, your best bet would be to go to either the developer of that plugin or a wordpress technical forum to see if anyone else has had a similar problem and come up with a solution.

[Martijn_Scheijbeler]

Agree with this, plus it seems that your robots.txt file isn't related as the files were temporarily unreachable and not being blocked by robots.txt.

[WallerD]

Thanks Hutch,

I have been trying to clean the site up and thought I had done something wrong somewhere.

Just fetched again and the plug in was not mentioned so all good?

Best wishes.

David

[WallerD]

Thank you Martijn!

David.

[jaintechnosoft]

HeY!!!

We have a coupons and deal website. Coupons are added and removed from the website on a daily basis. But crawler isn't crawling it that often. Lately we started fetching and rendering the page, but that is a time taking task as we have more than 500 stores with coupons.

So, I was looking for some API or some method using which the crawler would crawl the website as defined. Suppose "x" store page should be crawled every alternate day as we daily update the coupons there, whereas "y"  store coupons are update fortnightly so , they can be crawled weekly.

Can somebody suggest me something..