Block search bots on staging server
-
I want to block bots from all of our client sites on our staging server. Since robots.txt files can easily be copied over when moving a site to production, how can i block bots/crawlers from our staging server (at the server level), but still allow our clients to see/preview their site before launch?
-
Robots.txt should be sufficient, but if you want to completely block them, here is a useful resource: