480,000 Redirects - Is this hurting my SEO? PLEASE HELP!

[niconico101]

Hello everyone,

I have over 480,000 internal rewrites in my Magento site. The reason I have so many is because I have over 1,500 products on my site and I update inventory every day via Bulk Import Extension. For the first few months I didn't realize that the URL was changing by a single digit every time I imported the .xml with new inventory counts. This of course created thousands and thousands of 404s.

I figured out how to avoid the digit change and then I started redirecting the 404s via a Bulk Rewrite Extension. I managed to rewrite over over 50,000 404s but new ones still pop up every day and there is no end to them in sight.

My traffic is terrible. Only about 40 organics daily. It's been like that for months. I can't get it off the ground and I think it's because of this excessive rewrite and 404 issue.

My question is, does having so many internal rewrites and 404s hurt my SEO efforts?

Would it be better just to start from scratch with a new site, new domain, new everything?

Please help me. I'm going crazy with this. Thank you.

Nico.

[Everett]

Hello Nico.

I think I have some good news for you. Unless there is live link to those URLs (internal or external) there is no need to redirect them. In my experience more than 99% (an educated guess) of product pages do not have any external links. So assuming you have not cleaned up your internal linking (if you haven't, you should) there really is no need for the 301 and you can just let it resolve as a 404.

As long as the old URLs return a 404 status code they should be removed from the index within a matter of weeks. However, if they don't have any external or internal links - again, most of those pages probably do not - it may take months for Google to decide to recrawl an indexed page without any link paths to it. Googlebot couldn't just "crawl" its way there, but would have to retrieve the URL from a database just to check it, which would take longer. I have a tested and recommended solution to this issue, which can be found here on the Moz.com blog.

[niconico101]

This post is deleted!

[niconico101]

Hi Everett,

Thank you very much for your feedback.

This does put me at ease because I was ready to pull the plug and start a new site from scratch.

Quick question, do old Rewrites disappear after a while? In others words, do they keep getting crawled?

If so, can I deleted them after a few months for example, or after a year, or do search engines need them to exist permanently because other wise they would generate new 404s?

Thank you.

Nico.

[Everett]

Nico,

Great question. Technically speaking, a permanent 301 redirect should stay up indefinitely. However, I have heard more than one Google employee say publicly that they should stay up for "at least a year". At that point I think you can take them down and the destination page will retain the link juice from the old links since Google will have supposedly updated their link graph. However, I don't trust that most Google employees really know what's going on under the hood all the time, and recommend leaving any redirects from pages with external links in place for as long as you have control over it (e.g. you obviously couldn't leave them in place across domains if you sold the old domain).

If the page doesn't have any internal or external links pointing to it you can safely remove the 301 redirect at any time. A year would be plenty.

[niconico101]

Thank you Everett. You don't know how much this helps me. I was dreading having to start the site from scratch, so I will continue with the one I have and work on it for as along as it takes. This is the site by the way: http://devilswink.com/

Do you offer SEO consulting services for small businesses in the U.S. Everret?

[Everett]

Hello Nicolas,

You're welcome! I am glad you found the answer helpful. I direct the SEO strategy for seOverflow, and yes we do work with small businesses. Fill out our contact form and we'll be in touch. You can also look into other reputable SEO agencies and freelancers listed here on Moz.com.

Nice site.