Are 404 Errors a bad thing?

[Prime85]

Good Morning...

I am trying to clean up my e-commerce site and i created a lot of new categories for my parts...

I've made the old category pages (which have had their content removed) "hidden" to anyone who visits the site and starts browsing.  The only way you could get to those "hidden" pages is either by knowing the URLS that I used to use or if for some reason one of them is spidering in Google.

Since I'm trying to clean up the site and get rid of any duplicate content issues, would i be better served by adding those "hidden" pages that don't have much or any content to the Robots.txt file or should i just De-activate them so now even if you type the old URL you will get a 404 page...

In this case, are 404 pages bad?  You're typically not going to find those pages in the SERPS so the only way you'd land on these 404 pages is to know the old url i was using that has been disabled.

Please let me know if you guys think i should be 404'ing them or adding them to Robots.txt

Thanks

[Brian-H]

My understanding is 404 linked on your site is bad as it gives google bots deadends when what you really want is for them to crawl as much content as possible. As far as 404 on pages in it's index, not a problem as this is to be expected in an ecommerce site that products no longer exist etc. If a product has moved and still exists just under a different name or title you should 301 redirect. Especially so if that page has inbound links (found using openexplorer) but also because it helps people not reach dead ends when searching for things. In summary: If its moved 301 it. If it doesn't exist any more remove links from within your site to it and let it die.

[Joseph-Green-SEO]

Hi,

Here is what google says about 404s

"Q: Do the 404 errors reported in Webmaster Tools affect my site’s ranking?
A: 404s are a perfectly normal part of the web; the Internet is always changing, new content is born, old content dies, and when it dies it (ideally) returns a 404 HTTP response code. Search engines are aware of this; we have 404 errors on our own sites, as you can see above, and we find them all over the web. In fact, we actually prefer that, when you get rid of a page on your site, you make sure that it returns a proper 404 or 410 response code (rather than a “soft 404”). "

" If some URLs on your site 404, this fact alone does not hurt you or count against you in Google’s search results. However, there may be other reasons that you’d want to address certain types of 404s. For example, if some of the pages that 404 are pages you actually care about, you should look into why we’re seeing 404s when we crawl them! If you see a misspelling of a legitimate URL (www.example.com/awsome instead of www.example.com/awesome), it’s likely that someone intended to link to you and simply made a typo. Instead of returning a 404, you could 301 redirect the misspelled URL to the correct URL and capture the intended traffic from that link. You can also make sure that, when users do land on a 404 page on your site, you help them find what they were looking for rather than just saying “404 Not found."

http://googlewebmastercentral.blogspot.com/2011/05/do-404s-hurt-my-site.html

I would recommend you to 301 redirect them to the appropriated page  and 404 what you dont find a good page to redirect.

[Cyrus-Shepard]

Hello Prime85,

Both these answers are correct in their own way, but let me clarify and add my 2 cents.

1. 404s don't hurt your rankings directly, but they can provide a poor user experience.

2. If you keep URLs "live" - then Google can keep these URLs in their index indefinitely. This means search engines may waste time and crawling resources visiting pages you don't want in the index, while ignoring your other pages. This CAN hurt your SEO.

Long story short, (like Brian says) if the page is no longer relevant, you should remove it from the index or redirect it to another URL.

3. Returning a 404 kills all link juice that may have gone to the page, and it can also send confusing signals to search engines about the structure of your site if you have a bunch of pages returning 404s at the same time you have a bunch of new, but similar, pages popping into existence.

The best policy is to set up a 301 redirect from your outdated pages to the most relevant new pages. Don't redirect everything to a single page like the homepage, but instead the redirect to the page that would be most relevant and useful for the user.

On the other hand, if it's a low-value page and there's really no need to redirect it, you should remove it from the index. There's a couple ways to do this:

1. Put a meta robots "NOINDEX" tag in the head and wait for Google to crawl the page and process the noindex. It helps if the URL is listed in your sitemap so that they can more easily "find" the url.
2. Block the URL through robots.txt, then use Google's Remove URL tool in Webmaster tools
3. Return a 404, and use the Remove URL tool

Hope this helps! Best of luck with your SEO.