Can I Disallow Faceted Nav URLs - Robots.txt

[tylerfraser]

I have been disallowing /*? So I know that works without affecting crawling. I am wondering if I can disallow the faceted nav urls.

So disallow: /category.html/? /category2.html/? /category3.html/*?

To prevent the price faceted url from being cached:

/category.html?price=1%2C1000
and
/category.html?price=1%2C1000&product_material=88

Thanks!

[AlanMosley]

You can, but then you have links passing link juice to non followed pages. it would be better if you used canonical. even better would be to add no-index, follow meta tag when non canonical page is displayed, but this requres some codeing.

[tylerfraser]

If I can, then I disallow hundreds of pages that are duplicate content and should not be crawled.

If I don't then I send link juice to urls that I don't want seen.

This is a good answer though, thanks. Any other thoughts?

[Francisco_Meza]

Hey Tyler! haven't seen you on SEOmoz in a while. Hope you are good!

Check to see if this would make sense for you. GWT > Site Configuration > URL Perameters. It says "Only use this feature if you feel confident about how parameters work for your site. Telling Googlebot to exclude URLs with certain parameters could result in large numbers of your pages disappearing from our index."

[AlanMosley]

If you can no-index , follow all but the default, then you will send link juice to the pages but it will return the link juice because it is follow, but they will not index because they are no-index.

If you use robots, then it can not read the page to follow the links.