Will Google Revisit a 403 Page
-
Hi,
We've got some pretty strict anti-scraping logic in our website, and it seems we accidentally snared a Googlebot with it.
About 100 URL requests were responded to with a 403 Forbidden error. The logic has since been updated, so this should not happen again.
I was just wondering if/when Googlebot will come back and try those URLs again. They are linked from other pages on the site, and they are also in our sitemap.
Thanks in advance for any assistance.
-
My belief is that they always visit a page a few times before they make a decision for this reason.
If you have links to the page than they will certainly revisit. -
Thanks for the reply Alan. We'll wait for them to crawl the site again.