Can I use content from an existing site that is not up anymore?

[WilliamKammer]

301 redirect the old web pages to the new ones using an .htaccess file on the old website. This will show Google that the content has moved to the new web pages. Check out the link for more information: http://moz.com/learn/seo/redirection

[RoxBrock]

Many bad links were built on the old website by a questionable SEO firm, so I do believe the URL has been hit, but not with a formal penalty.

In order to redirect the old web pages I would need to keep the website live which really does not serve my purpose--which is to use great content that was written in-house on a clean website with no backlinks (starting from scratch).

How would one go about "de-indexing" content?

Thank you for prompt responses.

[WilliamKammer]

You would need to keep the site live to speed up the de-indexation. Then block all bots through robots.txt and force a crawl.

Make sure this is what you want to do. There are other options for this situation depending on your intent. Canonical tags, for example, would not transfer penalties and still show Google where the good source of the content is.

[RoxBrock]

So you suggest leaving the old site up and add the content to the new site with the canonical tag pointing to old site? Any other options you can think of?

[WilliamKammer]

Put the canonical tags in the old content, and point it to the new pages.

If you believe there are penalties, then 301ing is a little risky.

De-indexing content doesn't mean Google forgets it was there, they still have it cached, so this isn't ideal.

It looks like canonical may be your best bet.

[JaneCopland]

The idea of using canonicals interests me, but I am not 100% sure it is risk-free. It used to be the case that you could 301 penalised websites and remove the penalty (we're talking 2010 and earlier here). Google is very keen on transferring penalties these days, so I would be surprised if they are leaving a loophole for canonical tags open like this, or if they will keep that loophole open for long.

You would ideally leave the site live and remove its content as William says - once you see that the cached version of the site no longer contains the content you want to move, you can feel free to take the old site down and put the content up on the new site.

We don't know what lengths Google is going to or will go to to avoid people being able to re-use previously penalised content (including good content from penalised websites) but the safest thing you can do whilst using this old content right now is ensure the old content has been deindexed before putting it up again elsewhere.

The actual safest thing you can do is re-write the content, but I realise this might not be possible.

[WilliamKammer]

The thought behind canonicals is this:

• One of their uses is to fight against scrapers and such by still having the canonical tags in place when these spammy places grab your content.

• If penalties passed through canonicals, then the penalties these scrapers have would effect your site terribly. This is not the case, in my experience.

• So, unless Google has already implemented the human tracking that was discussed a few Whiteboard Fridays ago, this should work. And even with hardcore human tracking for penalities, I think its yet to be seen if this would focus on small sites trying to fix penalities as opposed to the large black hat spammers.

There is a bit of theorycrafting here, but in RoxBrock's specific situation, it looks like he has to pick the lesser of all evils.

[JaneCopland]

Really good point. Taking that into account, I might guess that an anti-manipulation method Google might employ is to grab registration details, hosting data, analytics codes, etc. and other identifying factors to determine whether the canonicalised content is owned by the same person. That is, canonicals between tightly-linked sites where the "duplicate" is penalised could hurt the canonical source, stopping people using this in place of the old 301 trick. If the scraper site has nothing in common with the source, Google does not pass on any negative metric from the duplicate.

This is just a theory too of course! I'd be confident assuming that they're taking precautions to stop this becoming a common trick. Awesome point!

[RoxBrock]

After reading Jane & William's discussion--do you both agree that canonicals is the way to go? The site will be similar (trying to create a non-penalized site). The sites will have different ip's and servers but a lot of the same content. None of the same backlinks... I just don't want to do the work if it's going to end up hurting me worse. I don't see how I can get all those bad backlinks removed.

[JaneCopland]

Hi there,

I would say that, taking William's point into account, canonicals might work in order to remove any possibility that Google would see the new site as copying the old one. That said, I can't guarantee that they could not either manually or automatically (manually would be much easier) note that the two sites are owned by the same person and that the domain change is a measure taken to avoid a penalty. The truly safest thing to do is to re-write the content and start afresh. The next safest is to remove the content from the old site, force a re-crawl / wait for Google to update its cache of the old site excluding the content, and then re-publish on the new site.

Canonicals will make this process quicker, but I don't believe it can be guaranteed that they won't result in Google making a stronger connection between the two sites, which might not go well. Again, this is only if there are enough similarities for Google to understand that this is not a scraper / scrapee situation but a situation where one entity owns both sites.

I'm sorry not to give a definitive answer.

[RoxBrock]

Thank you. That is a great answer!