Duplicate content issue with trailing / ?

[Richard555]

Hi ,I did a SEOmoz Crawl Test and found most pages show twice, for example:

A:  www.website.com/index.php/dog/walk

B:  www.website.com/index.php/dog/walk/

I've checked Google Analytics and 90% of organic search traffic arrives on the URLs with the trailing slash (B).

Question 1: Can I assume I've a duplicate content problem?

Question 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?

Question 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?

Kind regards and thank you in advance

Nigel

[Highland]

1. Google can generally tell the difference between pages that have syntactically similar URLs but it's considered a best practice to not make any engine do any guesswork whenever possible.

2. I would 301 one version just for uniformity but you should be fine as-is right now.

3. There's nothing wrong with that being in the URL. Google sees it as part of the URL and nothing more. I don't consider it aesthetic or user friendly but that's a different matter.

[donford]

Question 1: Can I assume I've a duplicate content problem?
-Yes

Question 2: Is it best to do 301 redirects from the 'non trailing slash' pages to the 'trailing slash pages'?
-Yes 301 is best, barring that use rel="canonical" on the page you want to index

Question 3: For some reason every web page has a '/index.php' in it (see A&B) above. No idea why. Should it be a SEO concern?

-Yes, this is a concern, use the same method to deal with the problem. Directories on the server side are usually assumed to have an index, if not the server can choose what to display, this can be very bad sometimes. As such most CMS content management systems fix the problem by generating content for the index.php or .html pages. However, there can be duplicate content issues since there are 2 urls with the same content, use 301 to get rid of the index.php at directory levels, or use canonical tags.

Hope that helps,

Don

[Richard555]

Thank you Highland & Donford.

Re my 3rd question, can I just clarify, should I  now 301 redirect both A & B  URLs to a new URL say www.website/com/dog/walk   ?

Many thanks!

[irvingw]

Yes, index.php should never show so 301 that plus the trailing slash to remove it

Ddefinitely canonical all of the pages to have the URL without the trailing slash

Make sure your sitemap xml files and internal linking structure does not have the trailing slash. if they do,, then fix them to reflect the proper URL

[Richard555]

Hi Irving

Thank you for your reply. You mention a good point regarding the sitemap.xml!

If I was to 301redirect pages A & B to a new page eg www.website.com/dog/walk/ then how would I also canonical A & B to the new page?

Surely once I have 301'd the A & B pages will be dead and redirecting traffic to the new page.

Kind regard and my apologies for any confusion.

Nigel

[donford]

Hi Nigel

You only need to 301 one of the pages, 301 is indicating a permanent move, so in the case you outlined above,

I would 301, A to B the decisions to use B was based soly off the value of the url you indicated.  If for any reason you prefer the url's not use trailing slash then use A.

It also would not hurt to add a canonical tag to B

To be clear here, whether you use

website.com/index.php/dog/walk

or

website.com/index.php/dog/walk/

Does not matter as far as SEO is concerned, I would make my decision based off of which url has the highest position in Google, and be consistent with this method throughout my site.

Hope that helps,