Duplicate content issue: staging urls has been indexed and need to know how to remove it from the serps

[Taiger]

duplicate content issue: staging url has been indexed by google ( many pages) and need to know how to remove them from the serps.

Bing sees the staging url as moved permanently

Google sees the staging urls (240 results) and redirects to the correct url Should I be concerned about duplicate content and request Google to remove the staging url removed

Thanks Guys

[Malika1]

I am assuming that these pages don't hold any authority or backlinks at all. You can simply delete these pages (if the purposes of these pages has been solved.

Or if you still need these pages live, use Robots.txt file to make these pages (or the whole subdomain/directory they are sitting as disallowed, no-index)

[Mobilio]

Quick way to remove staging url is sending HTTP error 410 as result.
Other is to use in SearchConsole Remove URLs function https://www.google.com/webmasters/tools/url-removal

About duplicate content - you must see actual canonical. If on stage URL there is canonical point to normal site then you shouldn't hesitating. But if staging and normal point to different URLs then you can see some algo filter.

[Taiger]

Thanks for your suggestions Peter and Malika,

By the wayt The staging site had it's own url..

I think I need help with the canonical stuff, as I am not really sure how to use it.

[ChrisAshton]

Generally you'll want to hide your staging site from search engines and as Malika mentioned, the best way to do this is via robots.txt.

That lets you essentially set a rule stating that no crawlers are to access anything on that domain. Beyond that, nothing else is really relevant; if crawlers can't see your site, it doesn't matter what you do with it! You don't even need to worry about 301 redirects once this is done.

Once you apply that change in robots.txt, you may still see your staging site indexed for a little while (anywhere from hours to a couple of months) but this is normal and it will drop away soon enough.

Search engines are pretty good at determining which is the real site these days anyway!

[evolvingSEO]

Thanks for helping Malika! To clarify for other readers, blocking in robots.txt after the pages have been indexed will actually prevent them from being removed from the index with a meta noindex tag, since Google won't be able to crawl the pages to see the noindex tag.

If staging URLs have been indexed already (and assuming they still need to exist), here's the steps I would take:

1. Add meta noindex tags to every staging URLs
2. If urgent, also do a URL removal request in Webmaster Tools (but this is usually not needed)
3. Wait until the staging URLs are noindexed - you can check periodically by doing site: searches in Google.
4. Only after they are noindexed, block Search Engines from crawling them with the robots.txt file.