Duplicate Titles caused by multiple variations of same URL

[adhunna]

Hi. Can you please advise how I can overcome this issue. Moz.com crawle is indicating I have 100's of Duplicate Title tag errors. However this is caused because many URL's have been indexed multiple times in Google. For example.

www.abc.com
www.abc.com/?b=123

www.abc.com/

www.abc.com/?b=654

www.abc.com/?b=875

www.abc.com/index.html

What can I do to stop this issue being reported as duplictae Titles, as well as content?

I was thinking maybe I can use Robots.txt to block various query string parameters. I'm Open to ideas and examples.

[JMacSupply]

I would redirect all variations to www.abc.com as well as REL=Canonical back to www.abc.com.   This should solve you issues.

[MattRoney]

*Edit: Miki beat me to it, but here's a little more explanation.

The first thing to note here is that Google's indexing doesn't actually have any effect on your Moz crawl report. All of the data you see there comes from our very own rogerbot, which crawls similarly to googlebot.

Though Google's crawler has a wide variety of ways to locate and index content, rogerbot can only crawl links on your site. If your crawl report is picking up each of these URLs, then there must be links pointing to those URLs somewhere on your site. The danger here is that Google and the other search engines will pick up those variants and not be able to determine which of them is the "real" one. That could lead to a) Google listing a URL you'd rather it didn't, or b) Google not understanding how to list your site at all.

A few of these have pretty simple fixes—index.html should be 301 redirected to your root domain, for example. Rel="canonical" is very applicable here, too. Here are a couple resources you may want to check out:

http://moz.com/learn/seo/canonicalization - Best practices article on canonicalization
http://moz.com/learn/seo/redirection - Best practices article on redirects

I hope that helps!

Matt Roney
Moz Customer Mentor

[adhunna]

Thank you for your response.

Does this mean for each main page I have i.e.

abc.com/page1

abc.com/page2

etc,

if I put a Rel="canonical", I can then ignore messages of duplicate content for URL's reported such as

abc.com/page1 (put a rel="canonical" on this page)

abc.com/page1?b=123

abc.com/page1?b=745

abc.com/page2 (put a rel="canonical" on this page)

abc.com/index.html (put a rel="canonical" on this page)

etc,

?

[adhunna]

This post is deleted!

[MattRoney]

Depending on how you implement the canonicals, you should see a decrease in your duplicate errors, which will be replaced by canonical notices. Ideally, there won't be anything to ignore.