Duplicate Forum Content

[LeeC]

HI everyone,

great to be here, absolutely loving everything, please go easy on me I'm quite a noob when it comes to seo, but hopefully my question isn't too basic.

After running the initial checks on my websites, I found there are 7,646 duplicate pages? Some are easy fixes but the majority are not, being forum pages, the edit, quote and new post links are coming up as duplicates of the main post?

Does anyone know how to fix this?

Best Lee

[Leonie-Kramer]

Hi Lee,

I'm not very familiar with forums. but i think you can solve it by putting a canonical in the head.

If you have a main page, which is the post, put a canonical url in it:

the duplicate pages need to have the same canonical.

I hope this work for you.

Grtz, Leonie

[LeeC]

Hi Leonie,

thanks for taking the time to answer

Not sure if that will work, there is only one head (hope that makes sense). Forums work pretty much the same as content management systems. I have the ability to change what's contained in the head, but adding rel="canonical" will have the same effect on all pages, even the duplicate one (I think).

Is there a way of removing all pages from the index (in webmaster tools) that contain the word edit, or quote or new? Kind of like using a wild card?

[Leonie-Kramer]

Hi Lee,

What kind of CMS are you using?

In wmt under parameters you can configure parameters. If the ur's contain the same parameter it is a posibility.

[LeeC]

is a paid forum service, but they give you the ability to add code snippets in the head.

[Leonie-Kramer]

Ah okay, that will be difficult than.

Maybe you can ask them to implement the canonical url. It will be helpful  for you to avoid duplicates

[LeeC]

or go somewhere else lol, have been considering buying invisionpower forum for a while, this might be the final push I needed.

Is there a way of mass deleting pages from the index in webmaster tools, i.e. all links that contain edit, quote and new ect?

[Leonie-Kramer]

Lol,  tha'ts also an option

under url's index, though they have to be removed first, otherwise google crawl them again. but there you can remove complete directories at once.

[LeeC]

Mmmm this is becoming more problematic by the minute, will need to implement robot.txt before removing the urls.

And there was me thinking this seo stuff would be easy.

[Leonie-Kramer]

Seo stuff is'nt that difficult, though it can be comlicated

Anyway I wish you luck with the forum and hope you'll manage to get it the way you wanted.

Grtz, Leonie

[LeeC]

Ok thanks Leonie,

still no nearer to knowing what to do, I know theres a problem and generally what I need to do, but I'm still at a complete loss as to how to do it

But thank you for trying to help,

Best, Lee

[Leonie-Kramer]

Hi, still one more thing, because you sound loss, and that's not the meaning of this Q&A. Contact the forumservice and ask for the canonical, this is a very simple and common thing to do.

[LeeC]

Hey

just found some valuable info which has solved a lot of my problems, modified .htaccess which has removed all of the non www links, so thats a part solve.

I think what I'm going to do is purchase the new forum software, setup a robot.txt to not crawl the new forums and then once I have all of the posts transfered to the new forum, remove the old forums from the index.

That should solve it, the new forum is very well established and if need be I can pay one of their developers to incorporate some seo feature that will stop the duplicate content issue. Once thats solved I can remove the robot.txt

Many thanks, Lee

[Ram.Babu]

invisionpower forum is good choice, as  i knew. Found duplicated content pages here, it happens with most of forums, but there is full provision that this can resolved by canonical or stop crawling one you don't want in robots.txt.

[LeeC]

How delightful that you refer to Invisionpower as I recently took the software for a test drive and plan to purchase it in January

A question though that has been troubling me. I use a paid for hosting service, which forwards to one of my sub-domains. The problem is I have nearly 7k pages of duplicate content (all from the forum), I really want to add a no follow and then remove the forum pages from the search index and thus remove all of the duplicate content. Though I'm worried that adding no follow to the sub domain may impact on the main site.

I have a lot of page one search positions, many long tails which combined result in a lot of visits, but I also have a couple of short phrases that get 50% of the visits alone, so I can't risk making any changes, actually I'm pretty terrified to make any changes, just in case all of my website pages are removed from the index.