Duplicate content /index.php/ issues

[b18turboef]

I'm having some duplicate content issues with Google.  I've already got my .htaccess file working just fine as far as I can tell.  Rewriting works great, and by using the site you'd never end up on a page with /index.php.  However I do notice that on ANY page of the site you could add /index.php and get the same page i.e.:

www.mysite.com/category/article

and

www.mysite.com/index.php/category/article

Would both return the same page.  How can I 301 or something similar all /index.php pages to the non index.php version?  I have no desire for any page on my site to have index.php in it, there is no use to it.  Having quite the hard time figuring this out.

Again this is basically just for the robots, the URL's the users see are perfect, never had an issue with that.  Just SEOMOZ reporting duplicate content and I've verified that to be true.

[ExactNiche.com]

Hi Emory,

Simple solution would be to redirect to root from the index.php using htaccess using the rule below. Lets us know how this works for you

RewriteRule ^(.*)index.(html|php)$ http://%{HTTP_HOST}/$1 [R=301,L]

[b18turboef]

Tried that in many ways, but can't get it working.  Here is a copy of the .htaccess file, what changes would need to be made (clearly input that code):

Options +FollowSymLinks

RewriteEngine On

prevents people from accessing anything with phpMyAdmin

RewriteRule phpMyAdmin - [F]

Remove index.php or index.htm/html from URL requests

#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(([^/]+/)*)index.(php|html?)\ HTTP/
#RewriteCond %{REQUEST_URI} !^/administrator
#RewriteRule ^([^/]+/)*index.(html?|php)$ http://www.mysite.com/$1 [R=301,L]

force canonical www if request is for non-www or has port number etc

RewriteCond %{HTTP_HOST} !^(www.mysite.com)?$
RewriteRule (.*) http://www.mysite.com/$1 [R=301,L]

redirect 301 /home.html http://www.mysite.com/

RewriteCond %{QUERY_STRING} mosConfig_[a-zA-Z_]{1,21}(=|%3D) [OR]
RewriteCond %{QUERY_STRING} base64_encode[^(]([^)]) [OR]
RewriteCond %{QUERY_STRING} (<|%3C)([^s]s)+cript.(>|%3E) [NC,OR]
RewriteCond %{QUERY_STRING} GLOBALS(=|[|%[0-9A-Z]{0,2}) [OR]
RewriteCond %{QUERY_STRING} _REQUEST(=|[|%[0-9A-Z]{0,2})
RewriteRule .* index.php [F]

#RewriteBase /
RewriteRule .* - [E=HTTP_AUTHORIZATION:%{HTTP:Authorization}]
RewriteCond %{REQUEST_URI} !^/index.php
RewriteCond %{REQUEST_URI} (/[^.]|.(php|html?|feed|pdf|raw))$ [NC]
RewriteCond %{REQUEST_FILENAME} !-f
RewriteCond %{REQUEST_FILENAME} !-d
RewriteRule . index.php [L]

[b18turboef]

Any ideas/input?

[ThompsonPaul]

Hey Emory - if that's the default .htaccess file your software created (assume this is a Joomla-based site?), it looks like the redirect code you need is already there, but it is disabled by default.

The following code

Remove index.php or index.htm/html from URL requests

#RewriteCond %{THE_REQUEST} ^[A-Z]{3,9}\ /(([^/]+/)*)index.(php|html?)\ HTTP/
#RewriteCond %{REQUEST_URI} !^/administrator
#RewriteRule ^([^/]+/)*index.(html?|php)$ http://www.mysite.com/$1 [R=301,L]

should do what you want, The reason its not currently doing anything is because it has been commented out. The "#" symbol at the beginning of each line tells the server NOT to run the code in that line.

Try removing the "#" symbol in front of the last three lines of that code, save the file & then thoroughly test your site. (It's not the way I would write it, but there may be specific requirements for your site/system) The first line is just a descriptive header, so the "#" symbol needs to be left on it.

If for any reason it causes problems, you can simply re-add the "#" symbols and re-save to return the site to its original state.

Give that a shot and let us know if it accomplishes what you want to do.

Paul

P.S. In particular when testing - ensure that client logins work correctly, and that the search function and all plugins also still work.