Meta Robots Noindex and Robots.txt File

[iskq]

Can we use meta robots noindex and robots.txt together?

[Devanur-Rafi]

Hi, let us take a scenario. test.html has been blocked using robots.txt file. In this scenario, the bots that respect the robots exclusion protocol will not crawl the page to encounter the page level robots noindex attribute. So does not make any sense to use both the page level robots exclusion method and server side robots.txt file.

Here you go for more:

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93708

And here:

http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710

Best regards,

Devanur Rafi.