Old pages still crawled by SE returning 404s. Better to put 301 or block with robots.txt ?

[H-FARM]

Hello guys,

A client of ours has thousand of pages returning 404 visibile on googl webmaster tools. These are all old pages which don't exist anymore but Google keeps on detecting them. These pages belong to sections of the site which don't exist anymore. They are not linked externally and didn't provide much value even when they existed

What do u suggest us to do:

(a) do nothing

(b) redirect all these URL/folders to the homepage through a 301

(c) block these pages through the robots.txt.

Are we inappropriately using part of the crawling budget set by Search Engines by not doing anything ?

thx

[RyanKent]

Hi Matteo.

The first step I would suggest is determining the source of the links to these 404 pages. If these links are internal to your website, they should be removed or updated.

The next step I would recommend is to ensure your site has a helpful 404 page. The page should offer your site's navigation along with a search function so users can locate relevant content on your site.

I realize that thousands of broken links may seem overwhelming. It is a mess which should be cleaned up. How you proceed is dependent upon how much you value SEO. If your ranking is important and you want to be the best, you will have someone investigate every link and make the appropriate adjustments such as 301 redirecting them to the most appropriate page on your site, or allowing the link to continue to the 404 page.

It's a search engine's job to help users find content. 404s are a natural part of the web. There is nothing inherently wrong with having some 404 pages. Having thousands of pages really shows your site has significant issues. Google's algorithms are not revealed publicly but it's logical to believe they may consider sites with a high percentage of 404 pages less trustworthy. This is my belief but not necessarily that of the SEO community.