Why are pages still showing in SERPs, despite being NOINDEXed for months?

[MadeLoud]

We have thousands of pages we're trying to have de-indexed in Google for months now.  They've all got . But they simply will not go away in the SERPs.

Here is just one example....

http://bitly.com/VutCFiIf you search this URL in Google, you will see that it is indexed, yet it's had  for many months.  This is just one example for thousands of pages, that will not get de-indexed.  Am I missing something here?  Does it have to do with using content="none" instead of content="noindex, follow"?

Any help is very much appreciated.

[eyepaq]

Hi,

In order to deindex you should use noindex as content=none also means nofollow. You do need to follow now in order to reach all other pages and see the no index tag and remove those from the index.

When you have all of them out of the index you can set the none back on.

This is the main reason "none" as attribute is not very wide in usage as "shooting yourself in the foot" with it it's easy.

On the otehr hand you need to see if google bot is actually reaching those pages:

• see if you don't have any robots.txt restrictions first

• see when google's bot last have a hit on any of the pages - that will give you a good idea and you can do a prediction.

If those pages are in the sup index you can wait for some time for Google bit to revisit.

One last note: build xml sitemaps with all of those pages and submit those via WMT - that will help at 100% to get those in front of the firing squad and also to be able to monitor those better.

Hope it helps.

[Bryan_Loconto]

We always use the noindex code in our robot.txt file.

[SanketPatel]

Hi Howard,

The page is in Google index because you are still linking to that page from your website. Here is the page from where that page links:

http://www.2mcctv.com/product_print-productinfoVeiluxVS70CDNRDhtml.html

As you are linking that page Google indexing the page. Google come to know about "noindex" tag before that he has already indexed it. Sorry for bad English.

Lindsay has written awesome post about it here:

http://www.seomoz.org/blog/robot-access-indexation-restriction-techniques-avoiding-conflicts

After reading above blog post, my all doubts about noindex, follow, robots.txt get clear.

Thanks Lindsay

[MadeLoud]

Ok, so, nofollow is stopping the page from being read at all?  I thought that nofollow just means the links on the page will not be followed.  Is meta nofollow essentially the same as blocking a page in robots.txt?

[eyepaq]

No, nofollow will only tell the bot that the page is a dead end - that the bot should not follow any links on page. And that means any inks from those pages won't be visited by the bot - that is slowing the crawling process overall for those pages.

If you block a page in robots.txt and the page is already in the index - that will remain in the index as the noindex or content=none won't be seen by the bot so it won't be removed from the index - it will just won't be visited anymore.

[edwardrj]

Just to add to the other answers, you can also remove the URLs (or entire directory if necessary) via the URL removal tool in Webmaster Tools, although Google prefers you to use it for emergencies of sorts (I've had no problems with it).

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=164734

[MadeLoud]

Hm, that is interesting.  So you're saying that it will get crawled, and thus will eventually become deindexed (as noindex is part of the content="none" directive), but since it's a dead end page, it just takes an extra long time for that particular page to get crawled?

[MadeLoud]

A page can have a link to it, and still not be indexed, so I disagree with you on that.

But thanks for using the domain name.  That will teach me to use a URL shortener...

[SanketPatel]

Thanks for your reply,

Let me know if you are able to deindex those pages. I will wait. Also please share what you have implemented to deindex those pages.