Dates in URL's

[carbisbayhols]

I have an issue of duplicate content errors and duplicate page titles which is penalising my site. This has arisen because a number of URLs are suffixed by date(s) and have been spidered . In principle I do not want any url with a suffixed date to be spidered.

Eg:-

www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/06_07_13/13_07_13

http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm/20_07_13/27_07_13

Only this URL should be spidered:-

http://www.carbisbayholidays.co.uk/carbis-bay/houses-in-carbis-bay/seaspray.htm

I have over 10,000 of these duplicates and firstly wish to remove them on block from Google ( not one by one ) and secondly wish to amend my robots.txt file so the URL's are not spidered.  I do not know the format for either.

Can anyone help please.

[KyleJack]

Hi Alan,

I would probably start by adding a disallow rule to robots.txt.

**Disallow: /*_**  _may work and block all your dated URLs from being indexed but may also have adverse affects if you have any URLs containing underscores. To test whether this solution would work I would firstly implement a disallow directly on a chosen dated URL, _**Disallow: /20_07_13 **_for example, and then test whether Google has noindexed the page. GWT should tell you whether you have inadvertently blocked any other pages by doing so.

You should also be thinking about how these URLs are being created and taking actions to prevent it. Consider implementing canonical tags if you haven't already to clean up any potential duplication issues.

Cheers,

K

[carbisbayhols]

Thanks Kyle.

Particularly grateful for the Disallow format, they are the only URL's using an underscore so will work for me.   WIll be checking why these are being created.

Do I need to remove them using the Removal Tool in Google, is there a format for doing this on block ?

Thanks again,

Alan