How should I handle URL's created by an internal search engine?

[GrappleAgency]

Hi,

I'm aware that internal search result URL's (www.example.co.uk/catalogsearch/result/?q=searchterm) should ideally be blocked using the robots.txt file. Unfortunately the damage has already been done and a large number of internal search result URL's have already been created and indexed by Google. I have double checked and these pages only account for approximately 1.5% of traffic per month.

Is there a way I can remove the internal search URL's that have already been indexed and then stop this from happening in the future, I presume the last part would be to disallow /catalogsearch/ in the robots.txt file.

Thanks

[LesleyPaone]

I am guessing that you are using a system that templates pages and maybe adds a query string after the search, something like search.php?caws+cars. I would set in the header of all of the pages that use the search template a noindex, nofollow. Then I would also add it to the robots text as well to disregard the search pages. They will start dropping out of the results pages in about a week or so.

[TheeDigital]

On top of Lesley's recommendations, both google and bing have url parameter exclusion options in webmaster tools.

[CleverPhD]

Basic cleanup

From a procedural standpoint, you want to first add the noindex meta tag to the search results first.  Google has to see that tag to then act on it and remove the URLs.  You can also enter some of the URLs into the Webmaster tools removal tool.

Next you would want to add /catalogsearch/ to robots.txt once you see all the pages getting out of the index.

Advanced cleanup

If any of these search result URLs are ranking and are landing pages in Google.  You may want to consider 301 redirecting those pages to the properly related category pages.

My 2 cents.  I only use the GWT parameter handler on parameters that I have to show to the search engines. I otherwise try to hide all those URLs from Google to help with crawl efficiency.

Note that it is really important that you do the work to find what pages/urls Google has cataloged to make sure you dont delete a page that is actually generating some traffic for you.  A landing page report from GA would help with this.

Cheers!