About duplicate content

[NorbertoMM]

Hi i'm a new guy around here, but i'm having this problem in my website.

Using de Seomoz tools i ran a camping to my website, in results i get to many errors for duplicate conten, for example,

http://www.mysite/blue/

http://www.mysite/blue/index.html, so my question is,

what is the best way to resolve this problem, use a 301 or use the rel canonical tag?

Wich url will be consider for main url,

Thanks for yor help.

[E-dreamz]

I would recommend a 301 redirect.

For the best SEO value you want to leave off the /index.html - espcially if that is your homepage.

Don't forget to redirect non-www to www.

[saibose]

Add a rel=canonical tag for the pages that have multiple URLs.

You can see some resources here.

http://www.google.com/support/webmasters/bin/answer.py?hl=en&answer=139394

If you use the rel= canonical tag you will have to do it for all pages with this issue. If the issue is widespread, you can consider a 301, but that wont be very effective for SEO purposes when compared to rel=canonical tag.

[StalkerB]

Something very similar came up earlier, best to 301 them as E-Dreamz says.

If you put this in your .htaccess file you should have all your pages as www. and the index.html will disappear.

RewriteEngine On Options +FollowSymLinks

RewriteCond %{HTTP_HOST} ^example.com
RewriteRule (.*) http://www.example.com/$1 [R=301,L]

RewriteCond %{THE_REQUEST} ^./index.html
RewriteRule ^(.)index.html$ http://www.example.com/$1 [R=301,L]

[StalkerB]

Sorry Saibose, disagree entirely, canonical is a band aid whereas 301 is a fix.

[NorbertoMM]

Thanks for your help guys.

I have another question, why in google websmaster tools i don't recieve any massage error about it?

[StalkerB]

The only place you would see something in WMT would be DIAGNOSTICS > HTML SUGGESTIONS, and it should show as duplicate title tag and meta description.

WMT wouldn't flag that up specifically because it's not something it really checks for.

[NorbertoMM]

Hi guys thank you so much for your help. I have another think, looking deep in the report i saw a duplicate page title, it is a section like products reviews, for example like this:

Url 1: Title 1 : I like the product - product reviw name of product

Url 2: Title 2 : How can i get the product - product reviw name of product

Why this is condider like a same title page, can somebody help me to clear this doubt.

And what can i do to resolve this problem.

Thanks

[KeriMorgret]

Hi Norberto,

Are you still having duplicate content errors, or did you clear this up? We're happy to help if you're still having any problems, just add a response to this thread.

[PerriCline]

A while back I had asked our hosting company to create 301 redirects in the htacces file for the same issue (www.mysite.com/index.html to www.mysite.com, www.mysite.com/products/index.html to www.mysite.com/products/ .....)  THe response I received was "redirecting .../index.html to ..../ won’t work.  They’re the same page.  Apache will get in an infinite loop and the page won’t load. "

Any help would be greatly appreciated since I have 36 instances of this happening on our site.

[KeriMorgret]

Hi Perri,

This is an older thread, and people may not see the new response if they're not subscribed to it.

You can certainly redirect the index.html to /. The above thread gives some help, as well as http://www.smartlabsoftware.com/howto/redirect-index-page.htm (though I don't know the age of that post and if it's for a current version of Apache).

I suggest opening a new question here with a title something like "redirecting /index.html to / in apache" and give your details in that question, with a link to the above URL and ask if this is still valid. A link to your site in the question, if you can give it, would also be great.

Thanks!

[davebrown1975]

If you are only talking about your home page, then yes setup a 301 redirect as others have shown for the entries already in googles index BUT a redirect itself can lose up to 10% of any link juice flowing to your index page.  And if you're building off site links, do you link to your root domain or the specific URL of your homepage?  My guess is the root, i.e. www.mysite.com so unless www.mysite.com is actually a different website to whats found at www.mysite.com/blue/ then I always strive to get my sites working without an initial redirect taking place when someone goes to www.mysite.com

Depending on you choice of webserver, you can specify what the default index page should be, in apache this is known as the 'DirectoryIndex'.

If you add the line

DirectoryIndex /blue/index.html

to your .htaccess (or even better apache site config if you can) then apache will serve that page WITHOUT the redirect ensuring any link juice to your route domain is not diluted.

Then just make sure any links on your own site  that point to you home page DO NOT point to /blue/ or /blue/index.html but simply to "/" or "http://www.mysite.com/"

[dunklea]

I don't think some of the responses in this thread have given you adequate information to solve your problem. 301's and rel canonical are there to solve two very different problems, and when used correctly, can solve a lot of different SEO problems.

In your example you have two URLs which I am going to assume have the exact same information on them. Classic duplicate content situation. Ideally, I think you would want to delete one of these pages and create a 301 to redirect any users and links to the other page. This will focus all your content and links onto a single page and your PR and rankings will rise. I would choose to keep the page that has the better keywords in the URL, and no, it doesn't matter if you have the .html at the end of the URL. With or without, the actual keywords in the URL are more important.

The use of rel="canonical" has a very different purpose. Say for whatever reason you want to keep both of your URL's even though they have the exact same content (testing conversation rates, for example). In this case you would use a rel="canonical" on the page you don't want to rank in the search engines, pointing to the page you do want to rank for.

On http://www.mysite/blue/index.html for example, you would create this tag:  <rel="canonical" href="http://www.mysite/blue/">eCommerce sites have to do this a lot.</rel="canonical">

Rel canonical should not be used when you're trying to move content from one URL to another. That's what 301s are for.

[emikoo]

Hi,

I get duplicate content notifications on the following URLs:

www.mydomain.nl

and

www.mydomain.nl**/**

The / cause almost all my pages to show up as duplicate content. How to fix these?

Thnxs for the help!