Are 17000+ Not Found (404) Pages OK?

[jombay]

Very soon, our website will go a rapid change which would result in us removing 95% or more old pages (Right now, our site has around 18000 pages indexed).

It's changing into something different (B2B from B2C) and hence our site design, content etc would change.

Even our blog section would have more than 90% of the content removed.

What would be the ideal scenario be?

• Remove all pages and let those links be 404 pages
• Remove all pages and 301 redirect them to the home page
• Remove all unwanted pages and 301 redirect them to a separate page explaining the change (Although it wouldn't be that relevant since our audience has completely changed)- I doubt it would be ideal since at some point, we'd need ot remove this page as well and again do another redirection

[TomRayner]

Hi there

17,000 is quite a lot. I would look at maybe redirecting some of the URLs and I would do this based on certain criteria.

First of all, it helps to have a complete list of your current URLs. Screaming Frog is a great tool for this and is free.

Once you have your URLs, go into your analytics data and see which pages are attracting users. Take a sample size of about 2-3 months. If you're using Google analytics, click on traffic sources -> sources -> all traffic on the left-hand side.

When the dashboard loads, next to the "Primary Dimension" click other, and from the drop down menu click traffic sources, then landing page.

Any page with more than 5 or 10 visitors could be one worth redirecting.  If these are pages that visitors might frequently use to get to your site, ensuring they are redirected might help to not interrupt their user journey.  A 404 might put them off and go elsewhere.

Next, I'd look at what pages you might want to save to keep your SEO "strength". Put your URL into OpenSiteExplorer and then once done, click on "top pages". We're interested in the "Inbound Links" column here.  Export the file into a CSV then sort the URL list in Excel by the Inbound Link total.  You can filter here the pages with less links, so for instance you could remove the pages with 3 inbound links or less.  It's a general way of doing things and isn't foolproof, but you will be left with a list of pages that could be getting decent PageRank/link equity.  Manually check those pages and their backlinks and if you think they're acceptable, make sure you put in a 301 redirect.

Anything that doesn't match either of these criteria I would leave for a 404.  You may be left with a lot, but Google knows that 404s are an accepted part of the course and won't penalise you for them. Check out this webmasters blog link.

Hope this helps with your decision making!

[jombay]

Hi Tom.. Thank you for your advice.

The thing is, we don't want to retain the users. They are not going to serve our cause anymore (We used to spend thousands of dollars every month on server costs just to keep up with teh load. now we are cutting it down- so unwanted users are not really something we want as it would result in load increase)

I'll surely follow your advice on OSE. The thing is, we have lot of link to the pages from scraper sites. I am not sure if it's worth keeping though.

[DarinPirkey]

Just to add a little to this great reply...

Here is how I would determine if it was worth my time to keep some of the old pages.

If the industry is the same but the end user is different, I would make EVERY attempt to keep those old pages.  AuthorRank will matter in the future if you can contribute that information into a particular rel=publisher then I think it will be totally worth the time.

If, however, the information has nothing to do with the industry, then I wouldn't even consider taking the time to figure all of this out.  I would have a kick ass 404 page to help people find your new stuff though.

Remember too that when you 301 redirect you do in fact loose some "link juice".  (I really hate that phrase)  So if the incoming links are of little to now value then a 301 will provide even less.

[Everett]

Mohit,

Tom's advice will help you determine which pages are worth redirecting and which should just go to a 404 page (which should be customized instead of the browser/host default, and should also return a 404 response code in the http header!). My guess is that pages with links only from scraper sites aren't going to pass the tests laid out by Tom and thus would just go to a 404 page. However, any that have decent external links would fit the criteria and would be candidates for a 301 redirect.