Site architecture change - +30,000 404's in GWT

[bjs2010]

So recently we decided to change the URL structure of our online e-commerce catalogue - to make it easier to maintain in the future.

But since the change, we have (partially expected) +30K 404's in GWT - when we did the change, I was doing 301 redirects from our Apache server logs but it's just escalated.

Should I be concerned of "plugging" these 404's, by either removing them via URL removal tool or carry on doing 301 redirections? It's quite labour intensive - no incoming links to most of these URL's, so is there any point?

Thanks,

Ben

[LynnPatchett]

Hi,

Are you sure these old urls are not being linked from somewhere (probably internally)? Maybe the sitemap.xml was forgotten and is pointing to all the old urls still? I think that for 404's to show in GWT there needs to be a link to them from somewhere, so in the first instance in GWT go to the 404s and have a look at where they are linked from (you can do this with moz reports also). If it is an internal page like a sitemap, or some forgotten menu/footer feature or similar that is still linking to old pages then yes you certainly want to clear this up! If this is the case, once you have fixed the internal linking issues you should have significantly reduced list of 404s and can then concentrate on these on a more case by case basis (assuming they are being triggered by external links).

Hope that helps!

[grasshopper]

Hi Ben,

The answer to your question boils down to usability and link equity:

• Usability: Did the old URLs get lots of Direct and Referring traffic?  E.g., do people have them bookmarked, type them directly into the address bar, or follow links from other sites?  If so, there's an argument to be made for 301 redirecting the old URLs to their equivalent, new URLs.  That makes for a much more seamless user experience, and increases the odds that visitors from these traffic sources will become customers, continue to be customers, etc.
• Link equity: When you look at a Top Pages report (in Google Webmaster Tools, Open Site Explorer, or ahrefs), how many of those most-linked and / or best-ranking pages are old product URLs?  If product URLs are showing up in these reports, they definitely require a 301 redirect to an equivalent, new URL so that link equity isn't lost.

However, if (as is common with a large number of ecommerce sites), your old product URLs got virtually zero Direct or Referring traffic, and had virtually zero deep links, then letting the URLs go 404 is just fine.  I think I remember a link churn report in the early days of LinkScape when they reported that something on the order of 80% of the URLs they had discovered would be 404 within a year.  URL churn is a part of the web.

If you decide not to 301 those old URLs, then you simply want to serve a really consistent signal to engines that they're gone, and not coming back.  Recently, JohnMu from Google suggested recently that there's a tiny difference in how Google treats 404 versus 410 response codes - 404s are often re-crawled (which leads to those 404 error reports in GWT), whereas 410 is treated as a more "permanent" indicator that the URL is gone for good, so 410s are removed from the index a tiny bit faster.  Read more: http://www.seroundtable.com/google-content-removal-16851.html

Hope that helps!