Robots.txt Question for E-Commerce Sites

[IcanAgency]

Hi All,

I have a couple of e-commerce clients and have a question about URLs.

When you perform a search on website all URLs contain a question mark, for example: /filter.aspx?search=blackout

I'm not sure that I want these indexed.

Could I be causing any harm/danger if I add this to the robots.txt file?  /*?

Any suggestions welcome!

Gavin

[Joe.Robison]

You're right on target, it's not a good idea to index search results. Google doesn't want to crawl or index other search results in its own search results. There are some exceptions for gigantico sites like Yelp or TripAdvisor when showing their search results pages are actually the best option, but if you're not at that level and especially if you're an ecommerce site, it's not recommended.

You wouldn't be harming anything by excluding search from your robots.txt file. In fact, many top sites exclude search results to preserve crawl capacity and for indexation reasons.

You'll also want to look at parameter handling in Search Console, this article from Google will get you started.