301 Redirect using rewrite rule in .htaccess

[Trigun]

Hi Alan,

I think it's now clear to me that they should be rewritten.  Thanks for pointing me to the right direction.

I have a classified site and in my .htaccess I have these rewrite rules by default

RewriteRule ^/?(new)/(1_day)/([0-9]+)/([^./\"'?#]+).html$  index.php?a=11&b=$3&c=65&d=4 [L] ##category newest 1day
RewriteRule ^/?(new)/(1_week)/([0-9]+)/([^./\"'?#]+).html$  index.php?a=11&b=$3&c=65&d=1 [L] ##category newest 1week
RewriteRule ^/?(new)/(2_weeks)/([0-9]+)/([^./\"'?#]+).html$  index.php?a=11&b=$3&c=65&d=2 [L] ##category newest 2weeks
RewriteRule ^/?(new)/(3_weeks)/([0-9]+)/([^./\"'?#]+).html$  index.php?a=11&b=$3&c=65&d=3 [L] ##category newest 3weeks

RewriteRule ^/?(new)/(1_day)/([0-9]+)/([^./\"'?#]+)/([0-9]+).html$  index.php?a=11&b=$3&c=65&d=4&page=$5 [L] ##category newest 1day pages
RewriteRule ^/?(new)/(1_week)/([0-9]+)/([^./\"'?#]+)/([0-9]+).html$  index.php?a=11&b=$3&c=65&d=1&page=$5 [L] ##category newest 1week pages
RewriteRule ^/?(new)/(2_weeks)/([0-9]+)/([^./\"'?#]+)/([0-9]+).html$  index.php?a=11&b=$3&c=65&d=2&page=$5 [L] ##category newest 2weeks pages
RewriteRule ^/?(new)/(3_weeks)/([0-9]+)/([^./\"'?#]+)/([0-9]+).html$  index.php?a=11&b=$3&c=65&d=3&page=$5 [L] ##category newest 3weeks pages

unfortunately, these rules could not handle all URLs of the same format with different variables like the following below

http://www.mysite.com/index.php?a=11&b=15&d=3&c=1

http://www.mysite.com/index.php?a=11&b=15&d=3&c=2

http://www.mysite.com/index.php?a=11&b=16&c=5&d=1

http://www.mysite.com/index.php?a=11&b=16&c=5&d=1&page=2

http://www.mysite.com/index.php?a=11&c=5&d=1&b=230

Any idea on how I can solve this problem to avoid duplicate content?

Thanks in advance...

[AlanMosley]

I work on microsoft servers, i dont use .htaccess

but this is the rule i woudl write to fix all urls stating with index.php, no mater what the querystreing

<rule name="DefaultRule" stopprocessing="true"><match url="^index.php"><action type="Redirect" url="/" appendquerystring="false"></action></match></rule>

but try this let me know if it works i have a few other ideas

RewriteRule ^/index.php  /  [R=301,L]

[RyanKent]

Hi Mchael.

Yes, you can use htaccess to rewrite or redirect the URL.

Where do these URLs presently lead to? If these URLs are duplicates for pages on your site, I would suggest using a 301 redirect to send the traffic to the proper URL rather then your home page.

If your server uses cPanel, there is a Redirect tool you can use. This tool makes the process of adding a redirect easier and safer then modifying your htaccess file. Your htaccess file controls various aspects of your site's security, accessibility and SEO. The slightest error can cause your site to instantly be inaccessible. I would not recommend making any changes to your htaccess file except by an experienced programmer. Even using the correct code in the wrong order can lead to problems.

[RyanKent]

htaccess rewrite rules are based on Regex expressions. Your current Regex rewrite rules can be modified to adjust for the specific URLs. You need to locate an experienced programmer to write the expressions for you.

[AlanMosley]

All he needs to do is keep a back up, and he can have as many tries as he wants. He simpley has to replace the file with his back up if he goes wrong.
There is little danger here.

[RyanKent]

I'm sorry but the idea of advising users without expertise to modify their htaccess file is completely reckless. The trial & error approach can easily lead to circumstances where the rule works some of the time but not always. Worse, it can negatively impact other rules and site security causing major problems.

Without knowing the details of the site involved, I tend to make the safe assumption the site is important and there are one or more people who's livelihoods depend on the site. Having worked with clients who have recovered from the damage caused by errors in htaccess files I will firmly share my experience that no one other then a qualified expert should ever touch the file. The potential for damage is very high.

[AlanMosley]

I have
been programming since before the internet came to be, I have studied Computer
Science at University and passed numerous Microsoft Certifications, and while I
would not discount study, it is my experience that I have never met a great
programmer that did not learn by trail and error, after all this is how you
become experienced. There is no danger in using a backup, RegEx does not work
sometimes and not others, it is not dynamic, it is a static peace of code. You
will not excel at SEO unless you learn these things. I am sure Michael is capable
of deciding if he wants to do it himself, he seems to have got a long way already.
It would seem to me he is learning quite quickly. You may suggest that you
would not try, but I don’t think it is correct to forbid others.

[RyanKent]

Alan, we will just have to disagree on this topic.

I too have studied Computer Science in college. I too have a wall filled with MS certifications. I too have been programming since before the internet and even before hard drives existed. I am only 40 but the first PC I used was an Atari 800 and the command to save my work was "csave" which stood for "cassette save". This was before even floppy disks were popular and data was saved to cassette tapes.

I certainly am not forbidding anyone from taking whatever action they deem fit. It is indeed up to Michael or any reader to assess what changes they are comfortable making for their site.

The point I am making is many people grow very comfortable in making changes to their website, especially SEO-related changes. It is relatively safe to do such. If you make a mistake, your site may not rank as well, may not load as fast, may not appear correctly in all browsers and so forth. The consequences are relatively low.

Making changes in an htaccess file is a completely different ballgame. One character out of place and your site can instantly be taken off line. If that happened, it's actually not so bad compared to other problems which can be created. A character out of place can disable your site security and the person making the change would likely not realize the problem until their site was hacked. A character out of place can cause other functionality of your site to not work correctly. It can also cause the fix being implemented to work in some but not all instances.

I highly encourage users to make most changes to their sites according to their comfort level. Htaccess modifications is a clear exception. A user can easily be mislead to believe their site is working fine only to later realize there is a major problem with the site. There are countless instances where a site was exploited due to a vulnerability in the htaccess file. I therefore strongly recommend for users never to touch their htaccess file unless they are extremely confident in the changes they are making. Many websites will offer code snippets which can provide users a false sense of security and lead them to experiment. It is a bad idea to do such with the htaccess file.

[AlanMosley]

I disagree more with the level of apprehension, rather than the premise itself. Anyhow I’m off to bed.

[ShaMenz]

Hi Michael,

You do not need to make any changes to your .htaccess file. Actually, if you 301 these URLs you will break your search so that it no longer works.

The solution I would use is to go into Google Webmaster Tools and tell Googlebot to ignore the parameters you are concerned about.

In your code, the ? says "here come some parameters" and the & separates those parameters. So, in the case you have quoted, the parameters are a, b, c, d.

Be aware of course, that Roger will still see these URLs as duplicates since he doesn't know about your private conversations with Google   This means that they will still appear in your SEOmoz report, but as long as you make a note of them so you know they can be ignored that shouldn't be a problem.

Hope that helps,

Sha

[CraigBradford]

I have to say I agree with Sha on this one.

If you are not confident in using .htaccess then I wouldn't bother. I think there is a much easier solution:

1- As Sha said, use webmaster tools to tell Google how to handle these parameters, this should slowly start to take them out of the index.

2- Add rel=canonical to all your pages, this way even if parameters are added, the rel=canonical will always point back to the original and remove any risk of duplicate content.

I hope this helps.

Craig

[AlanMosley]

From memory, I believe Michael has these urls produced by his CMS and are unnecessary, i could be getting him mixed up with someone else.
also doing this in Google does not help other search engines, you would need to do it in all search engines for all possible combinations for each pages, this can become un-manageable.

[AlanMosley]

This is a solutions, but its an ugly one, does anyone really wants a home url of http://www.mysite.com/index.php?a=11&b=16&c=5&d=1&page=2. you then have the problem of people linking to that page.

I believe michael said in a previous post that they were prodused by his CMS, the best idea would be to get rid of them rather then deal wioth them if posible.