How to remove duplicate content, which is still indexed, but not linked to anymore?

[rmvw]

Dear community

A bug in the tool, which we use to create search-engine-friendly URLs (sh404sef) changed our whole URL-structure overnight, and we only noticed after Google already indexed the page.

Now, we have a massive duplicate content issue, causing a harsh drop in rankings. Webmaster Tools shows over 1,000 duplicate title tags, so I don't think, Google understands what is going on.

<code>Right URL: abc.com/price/sharp-ah-l13-12000-btu.html
Wrong URL: abc.com/item/sharp-l-series-ahl13-12000-btu.html (created by mistake)</code> 

After that, we ...

• Changed back all URLs to the "Right URLs"
• Set up a 301-redirect for all "Wrong URLs" a few days later

Now, still a massive amount of pages is in the index twice. As we do not link internally to the "Wrong URLs" anymore, I am not sure, if Google will re-crawl them very soon.

What can we do to solve this issue and tell Google, that all the "Wrong URLs" now redirect to the "Right URLs"?

Best, David

[SanketPatel]

Hi Davin

The best thing in this situation is to wait for sometime more.. Because you just done the redirection of wrong url's to right url's so it will take some time. In webmaster tool you will see the changes later because the data in webmaster tool are updates on 15 days or monthly basis, depends on the website so you need to wait. The url that was 301 redirected should not appear in the search results so the problem of duplication will be sorted out shortly so dont worry. Also you can verify the redirection are done correctly or not from this redirect checker tool http://www.internetofficer.com/seo-tool/redirect-check/.

I have one suggestion to crawl your website pages fastly : Maximize the "Crawl Rate" under Settings option of webmaster tool.

Hope my response would help you. If need any help feel free to ask.

[rmvw]

Thanks a lot, Sanket.

Do you think, it might help, to submit a sitemap, which also contains the "Wrong URLs", so we can trigger a recrawl of those pages? Maybe then Google will notice that there is a 301-redirect.

[rmvw]

Found the perfect answer:

http://www.seomoz.org/blog/uncrawled-301s-a-quick-fix-for-when-relaunches-go-too-well

[SanketPatel]

Yes David your link is very helpful..