Increase in Crawl Errors

[abisti2]

I had a problem with a lot of crawl errors (on Google Search Console) a while back, due to the removal of a shopping cart. I thought I'd dealt with this & Google seemed to agree (see attached pic), but now they're all back with a vengeance!

The crawl errors are all the old shop pages that I thought I'd made clear weren't there anymore. The sitemaps (using Yoast on Wordpress to generate these) all updated 16 Aug but the increase didn't happen till 18-20.

How do I make it clear to Google that these pages are gone forever?

Screen-Shot-2016-08-22-at-10.19.05.png

[Martijn_Scheijbeler]

HI,

Did you redirect the old pages to another page, like worst case the home page? The alternative would be to look into excluding these pages if they have a similar page path from the search engines by adding them to your robots.txt.

Martijn.

[abisti2]

Hi Martijn

Many thanks for your response. We have been redirecting old product URLs to the shop URL (not the front page, but the front page of the shop)  but I think ideally we just want rid of all these pages as the client is considering getting rid of the shop completely.

What would we do with robots.txt in this instance? Is there a tutorial you could point me at?

[BritneyMuller]

This is a great robots.txt cheat sheet to help figure out what robot exclusions might be right for your pages.

[abisti2]

Many thanks Britney