Which pages to "noindex"

[mmaes]

I have read through the many articles regarding the use of Meta Noindex, but what I haven't been able to find is a clear explanation of when, why or what to use this on.

I'm thinking that it would be appropriate to use it on:

legal pages such as privacy policy and terms of use
search results page
blog archive and category pages

Thanks for any insight of this.

[Theo-NL]

Noindexed pages are pages that you want your link juices flowing through, but not have them rank as individual entries in the search engines.

• I think your legal pages should rank as individual pages. If I wanted to find your privacy policy and searched for 'privacy policy company name', I'd expect to find an entry where I can click and find your privacy policy

• Your search results page (the internal ones) are great candidates for a noindex attribute. If a search engine robot happens to stumble upon one (via a link from somebody else for example), you'd want the spider to start crawling pages from there and spreading link juice over your site. However, under most circumstances you don't want this result page to rank on itself in the search engines, as it usually offers thin value to your visitors

• Blog archive and category pages are useful pages to visitors and I personally wouldn't noindex these

Bonus: your paginated results ('page 2+ in a result set that has multiple pages') are great candidates for noindex. It'll keep the juices running, without having all these pretty much meaningless (and highly dynamic) pages in the search index.

[CPU]

As well as pagination pages I have read, but not done it myself, that you should consider using it on low value pages that you are wouldn't want to rank above other pages on the site (hopefully they wouldn't anyway) and also sitemaps as don't necessarily want them to appear in the index but definitely want them followed.

[TellThemEverything]

yes, generally you would noindex your about us, contact us, privacy, terms pages since these are rarely searched and in fact are so heavily linked to internally that they would rank well if indexed.

all search results should be noindexed - google wants to do the search

definitely NOT blog/category pages - these are your gold content!

I also noindex any URL accessed by https

[KeriMorgret]

You also want to block any admin directory, plugin directory, etc. Are you using Wordpress or a specific CMS? There are often best-practice posts for robots.txt files for specific platforms.

[mmaes]

Yes I'm using wordpress.

[KeriMorgret]

Here are two posts that may be helpful in both explaining how to set up a robots.txt for wordpress, and the thinking behind setting up which parts to exclude.

http://www.cogentos.com/bloggers-guide-to-using-robotstxt-and-robots-meta-tags-to-optimise-indexing/

http://codex.wordpress.org/Search_Engine_Optimization_for_WordPress#Robots.txt_Optimization

The wordpress link (second link) has a link to several other resources as well.