Duplicate blog URLs in Magenton

[ahyde]

On one my sites Moz is picking up 4483 duplicate content pages. The majority of these are from our blog and video sections on our site.

We're using a URL shortener and it appears that some of the pages are the full version of the URL then the shortened version. However if you go to the full version you get redirected to the shorter one. So I would assume that the Moz crawler should get the same redirect?

We're also getting pagination being shown as duplicate pages, which I would half expect, but the URLs Magento is creating are truly bizarre:

e.g http://www.xxx.com/uk/blog/cat/view/identifier/news/page/news/index.php/alarms-doorbells/?p=2

Alarms and doorbells is one of our product categories, which is displayed in the LHN on the blog page but has nothing to do with the blog itself.

On another site on the same Magento instance, with the same content (they're for two different regions) we're show as having 248 duplicate pages, again in the video and news section, but this is a completely different scale of issue.

Has anyone else encountered issues like these? I'm probably going to put a noindex in place on these two sections until we can get a solution in place as we're completely unranked in google on this site.

Thanks

[seovp]

Hi,

Have you tried setting url parameters in webmaster tools? This can help you to tell Google which urls not to crawl depending on certain parameters, such as if the url has "?p"

[ahyde]

I actually had a look at the robots.txt last night to block the effected areas and discovered the dev's who built the site hadn't set it up. We've added a default one which we're going to test over the next week to see how it fares. If that doesn't work then we'll start looking at the parameters.

Thanks.