Indexation question

[RuudHeijnen]

Hi Guys, i have a small problem with our development website. Our development website is website.dev.website.nl This page shouldn't be indexed bij Google but unfortunately it is.

What can i do to deindex it and ask google not to index this website. In the robots.txt or are there better ways to do this?

Kind regards Ruud

[TomRayner]

Hi Ruud

As you've said, I'd go ahead and add a robots.txt file.  One thing to remember is that each subdomain requires its own robots.txt file.

In addition, I would add "noindex, follow" to the header of the pages you don't want crawl.

You can find out the best way to implement these changes with the Moz Robotstxt guide.

[irvingw]

In addition to Tom's response, you can also use Google Webmaster to remove the site out of the index a little faster:

Follow these steps to remove a site or directory from Google's search results:

1. On the Webmaster Tools home page, click the site you want.
2. On the left-hand menu, click Optimization and then click Remove URLs.
3. Click New removal request.
4. Type the URL of the directory you want removed from search results and then click Continue. (How to find the right URL.)
   • The URL is case-sensitive—use exactly the same characters and capitalization that the site uses.
   • If you want to remove the whole site, you can leave this blank.
5. Click Remove directory.
6. Click Submit Request.

Make sure you aren't accidentally linking to the dev site from your live site. I have seen this happen too many times.