How to Solve Duplicate Page Content Issue?

[Gustav-Northclick]

Hi!

Good question, one I often come across

I would say this is not a real issue for you in the serp. But if you want to fix it and competition is fierce in your field you should of course do it

Step one: Look up your index at google: simply with site:"youdomain.com" and see if there are any duplicated content in the index(chance is slim)

Step two: look up your webshop cms, perhaps there is a function for redirecting dynamic pages to a static page(usually there is)

Step three: If it's not possible to do the 301:s from webshop admin use the .htaccess file and implement 301:s with the url:s you have in the list.

Other comments:

If you would like to clean up these URLs work with 301:s and decide which structure is most important.(base it on the current index)

There are several ways to do an 301: but in this case I would say the easiest way since you have the list is to go to the .htacces and put the 301 redirect list there.

This way could of course be bad for you if the duplicated pages are somehow necessary for the webshop so start to look at the webshop to make certain that you don't mess up any critical part by doing it by .htaccess.

Best regards!

/ Gustav

[Dan-Petrovic]

Best practice in your case would be to implement URL canonicalisation (rel="canonical").

Watch this: http://www.google.com/support/webmasters/bin/answer.py?answer=139394 (contains explanation and examples)

In addition to this try to prevent page duplication form happening in the first place, though this may need to be done on a programming level.

It seems that string such as "6_129_130" appears based on the category and navigational path. For example if user browses from outdoor and home decor and arrive on the same page the URL will have different number.

In addition you may want to remove "zero products" pages from index and not link to them as they are not good for users or search engines.

[CommercePundit]

I am quite confuse with 301 redirect. There are too many duplicate pages which were generated due to wrong categorization or create new same category rather edit old category.

I want to remove all duplicate pages from my website and want to set 301 redirect.

I want to give one example for it.

Original Page:

http://www.vistastores.com/125_126/kitchen-and-dining/cookwares.html

Duplicate Page:

http://www.vistastores.com/126/cookwares.html

Duplicate pages were created due to Admin issue or by human error. No one will find our land on duplicate page during website surfing.

Now, I want to detect and remove all pages which are available with duplicate content.

So, Does it matter to delete that pages from website and set 301 to associated page or home page?

[Gustav-Northclick]

hi again!

After reading you follow up question I know a better solution for you

Instead of going canonical which takes much manual labor or doing 301:s since both are simply just temporary fixes for the problem and not the cure.

The best way and what I always recommend our clients is working with URL rewrite. This takes care of this problem completely, however it takes some coding to implement.

If you can handle this yourself and only need hints and guides read below:

• I assume that you use Apache server?
• If so read this: http://httpd.apache.org/docs/2.0/misc/rewriteguide.html

Or:

• Contact the admin for the webshop and ask them to make an URL Rewrite based on the URL structure that you would want to use
• Good structure:  root/productcategory/Product

Hope this helped you

[CommercePundit]

I think, This issue is not regarding URL rewriting. I can say very sure for it. As I said, duplicate page content URLs are not available on website. If any visitors will visit website and go to each page so that visitor will not able to find duplicate page.

URL rewriting will fix issue regarding URL structure but, what about duplication which is available in my own website to compete both pages with same keyword?

[CommercePundit]

In addition you may want to remove "zero products" pages from index and not link to them as they are not good for users or search engines.

Yes, you are right. I want to remove zero product pages as well as all pages which were created due to human error.

As I mentioned above: Following page is not available on website for buyers. Buyers will not able to see this page any more because, not a single page available on website which is available with hyperlink to that page.

http://www.vistastores.com/126/cookwares.html

But, SEOmoz crawler detect it and when I see in excel so I quite confuse. Because, that page is working and visible with all products which are available to original page.

Users are not able to go that duplicate page but Google crawler can go there and able to detect as duplication.

I have bit knowledge about duplication and assume that Google will detect both page as duplicate. That's it. And, I want to resolve it.

[Gustav-Northclick]

Ok!  Then we have that cleared

If you already are using URL rewrite there shouldn't be any duplicated content in the manner the list shows.

• But since there is a problem I would check so that you're not using a sitemap that dynamically crawls and creates these urls for the pages.

If that's not the issue:

• We then come back to doing the 301:s by using .htaccess.

/ Gustav

[Gustav-Northclick]

The tool you used to find these pages seems to work fine ;).

It's simply a crawl so use the seomoz data as it's a crawl of your site

Set a 301 redirect with the complete list of duplicated content, you don't need to delete the pages(it's not deleted anyway since it's dynamically created pages)

/ G

[CommercePundit]

Are you talking about HTML sitemap or XML sitemap?

If HTML sitemap so you are right and I suppose to make it live very soon with proper structure.

If you are talking about XML sitemap so I have created it with two partition.

http://www.vistastores.com/main_sitemap.xml

http://www.vistastores.com/products_sitemap.xml

So, What you think about it?

301 redirection is final. Right?? After removal of duplicate pages.

[CommercePundit]

Oh great. That's fine. Now, I got idea... What's wrong with my site. If I suppose to find that duplicate page in Admin so I may not able to find that pages. Right?? So, I just need to set 301 (Permanent Redirection) with help of htaccess. Now, It's clear. Thanks a lot for your prompt reply and quick discussion on my issue.

[Gustav-Northclick]

No probs glad to help!

Best of luck!