Duplicate Page Content / Titles Help

[gaz3342]

Hi guys,

My SEOmoz crawl diagnostics throw up thousands of Dup Page Content / Title errors which are mostly from the forum attached to my website.

In-particular it's the forum user's profiles that are causing the issue, below is a sample of the URLs that are being penalised:

http://www.mywebsite.com/subfolder/myforum/pop_profile.asp?mode=display&id=1308

I thought that by adding - _**http://www.mywebsite.com/subfolder/myforum/pop_profile.asp**_ to my robots.txt file under 'Ignore' would cause the bots to overlook the thousands of profile pages but the latest SEOmoz crawl still picks them up.

My question is, how can I get the bots to ignore these profile pages (they don't contain any useful content) and how much will this be affecting my rankings (bearing in mind I have thousands of errors for dup content and dup page titles).

Thanks guys

Gareth

[AgentsofValue]

You could use a wildcard operator in your robots.txt to deal with this.

Something like this should work:

Disallow: /subfolder/myforum/pop_profile.asp?mode=display&id=*

See here for more details:

http://sanzon.wordpress.com/2008/04/29/advanced-usage-of-robotstxt-w-querystrings/

[gaz3342]

Fantastic!

Thanks for this Agents of Value.

Gareth

[gaz3342]

Hi,

Just had my latest crawl test completed and unfortunately I am still getting thousands of Dup Page Title / Content errors.

It seems that adding /subfolder/my__forum/pop_profile.asp?mode=display&id=* to my disallowed list hasn't worked.

Can you think of anyting else I can try?

Thanks guys

Gareth