Removing/ Redirecting bad URL's from main domain

[redcappi]

Our users create content for which we host on a seperate URL for a web version. Originally this was hosted on our main domain.

This was causing problems because Google was seeing all these different types of content on our main domain. The page content was all over the place and (we think) may have harmed our main domain reputation.

About a month ago, we added a robots.txt to block those URL's in that particular folder, so that Google doesn't crawl those pages and ignores it in the SERP.

We now went a step further and are now redirecting (301 redirect) all those user created URL's to a totally brand new domain (not affiliated with our brand or main domain).

This should have been done from the beginning, but it wasn't.

Any suggestions on how can we remove all those original URL's and make Google see them as not affiliated with main domain?? or should we just give it the good ol' time recipe for it to fix itself??

[ThompsonPaul]

You need to get rid of the robots.txt block on those URLs you want to redirect, Alec.

As it is now with the robots block in place, you've told the search engines NOT to crawl those URLs So it's going to be very difficult for them to discover the 301 redirects and learn that they should be dropping the old URLs form the index. After that, it is just a matter of time. (It can also help to leave those old URLs in the xml sitemap for a while to make it easier for the engines to crawl them and discover the 301s)

If none of those URLs were generating any substantial amount of traffic or incoming links, you could also use Google and Bing Webmaster Tools to request that the pages be removed from the index. This will only really work if the pages are organised in a specific directory, as it would likely take far too long to annotate each URL for removal otherwise.

Hope that helps?

Paul

[Kurt_Steinbrueck]

It sounds to me like you don't want the search engines to know that your moving the content, but rather have them think that you have dropped the pages from your site because you don't want the search engines associating those pages with your site, correct?

If that's the case, then you do want to keep the noindex on the old pages and setup 301 redirects as well.  The redirects are for real users who happen to use any links/bookmarks to the old pages.  By keeping the old pages noindexed, then hopefully the search engines won't crawl them and won't follow the redirects.  I'd also remove the pages from the Google and Bing indexes in their webmaster tools for good measure.

If you are linking from your site to the new location of the user content, you may want to nofollow those links or, better yet, create the links in javascript or something to hide them.  If all the links to the content just shift to the new location, Google and Bing may still associate it your site with the new site.  Then again, if all the content from the old pages is all the new site, then they may figure it all out anyway.

[redcappi]

Yes, very helpful... Thanks!

[redcappi]

Yes, that's correct Kurt. We want to disassociate our brand from those pages. Thanks for your FB!