Google has indexed a lot of test pages/junk from the development days.
-
With hind site I understand that this could have been avoided if robots.txt was configured properly.
My website is www.clearvisas.com, and is indexed with both the www subdomain and with out.
When I run site:clearvisas.com in Google I get 1,330 - All junk from the development days.
But when I run site:www.clearvisas.com in Google I get 66 - these results all post development and more in line with what I wanted to be indexed.
Will 1,330 junk pages hurt my seo?
Is it possible to de-index them and should I?
If the answer is yes to any of the questions how should I proceed?
Kind regards,
Fuad
-
Go to Google Webmaster Tools => Optimization => Remove URLS
In order for Google to remove the URL, you will need to do 1 of the following:
1. Block it with robots.txt, but it sounds like it's too late for that.
2. If you removed the old development content, make sure that the old content's URL produces a 404 or 410 status code.
3. Block the content with a Meta noncontent tag.
In my opinion, option 2 is the easiest since you should have a 404 page anyway.
-
I would suggest following option 3 from David's recommendations.
Simply add the "noindex" tag to the pages you want removed from Google. The pages will then be removed the next time they are crawled.
You are correct the issue could have been avoided by blocking the site during development, which is a recommended practice; however, it is recommended to minimize entries in the robots.txt file of a live site. You can add the pages in robots.txt and Google can still index them.
The above applies if you feel the need to keep the pages around. If you no longer need those pages, removing them and providing a 410 error (GONE) would be the best approach.
-
Thanks for your view David, its much appreciated. Thanks, Fuad
-
Thanks Ryan.
Are the un-wanted/irrelevant pages likely to affect my organic seo?
-
It's impossible to say conclusively without examining your site and the content; however, since you refer to them as "junk" pages, it is likely they should best be removed to protect your other pages.
-
Thanks Ryan.