Getting subdomains unindexed

[kchandler]

If i turn an application off displaying a 503 error will that get my site unindexed from search engines?

[john4math]

Eventually, but that's the code Google recommends to return when your site is having downtime, so I would expect them to be more lenient towards not removing things right away.  I wouldn't expect it to be as efficient as returning a 404 or a 410.

The best way to get content de-indexed is to return a page with a meta noindex tag on it, if you're really keen on getting it removed immediately.

[kchandler]

Thanks for the quick reply, i will have to try that. Essentially i am trying to get the site un-indexed but i wasn't sure if a 503 would do the trick.

[SL_SEM]

No index tag on it works, and putting up a robots.txt that disallows everyone should work as well.

[KeriMorgret]

With a robots.txt blocking it, you can then go into Google Webmaster Tools and request removal of that particular page or folder from Google's index.

[kchandler]

GWT was the first placed i checked unfortunately you can only remove directories or pages. I need entire subdomained sites to be removed (in fact they shouldn't of been indexed in the first place).

We use subdomains for our development testing environment when creating client sites and once the site is approved we push it live replacing the old site. Somehow these testing sites are getting indexed and it may pose a threat to duplicate content on different domains. So i am trying to find a solution to get the subdomains (100's of them) unindexed.

I understand a 301 redirect is best but that isn't really applicable since these test sites still need to be reached by clients.

[KeriMorgret]

Subdomains can be verified as their own site in GWT. Verify the subdomain in GWT, then put a robots.txt on that subdomain excluding the entire subdomain, then request removal in GWT of that entire subdomain. I've had to remove staging and dev sites a couple of times myself.

A couple of things I've found useful in this situation is to make the robots.txt files for both the dev and live sites read only, so you don't accidentally overwrite one with the other when pushing a site live. You can also sign up for a free tool like Pole Position's Code Monitor that will look at the code of a page (including your robots.txt url) once a day and email you if there are any changes so you can fix the file then go hunt down whoever changed the file.