The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Search Engine Trends
    4. Google has indexed a lot of test pages/junk from the development days.

    Google has indexed a lot of test pages/junk from the development days.

    Search Engine Trends
    7 3 495
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Fuad_YK
      Fuad_YK last edited by

      With hind site I understand that this could have been avoided if robots.txt was configured properly.

      My website is www.clearvisas.com, and is indexed with both the www subdomain and with out.

      When I run site:clearvisas.com in Google I get 1,330 - All junk from the development days.

      But when I run site:www.clearvisas.com in Google I get 66 - these results all post development and more in line with what I wanted to be indexed.

      Will 1,330 junk pages hurt my seo?

      Is it possible to de-index them and should I?

      If the answer is yes to any of the questions how should I proceed?

      Kind regards,

      Fuad

      1 Reply Last reply Reply Quote 0
      • ToughTimesLawyer
        ToughTimesLawyer last edited by

        Go to Google Webmaster Tools => Optimization => Remove URLS

        In order for Google to remove the URL, you will need to do 1 of the following:

        1.  Block it with robots.txt, but it sounds like it's too late for that.

        2.  If you removed the old development content, make sure that the old content's URL produces a 404 or 410 status code.

        3.  Block the content with a Meta noncontent tag.

        In my opinion, option 2 is the easiest since you should have a 404 page anyway.

        Fuad_YK 1 Reply Last reply Reply Quote 1
        • RyanKent
          RyanKent last edited by

          I would suggest following option 3 from David's recommendations.

          Simply add the "noindex" tag to the pages you want removed from Google. The pages will then be removed the next time they are crawled.

          You are correct the issue could have been avoided by blocking the site during development, which is a recommended practice; however, it is recommended to minimize entries in the robots.txt file of a live site. You can add the pages in robots.txt and Google can still index them.

          The above applies if you feel the need to keep the pages around. If you no longer need those pages, removing them and providing a 410 error (GONE) would be the best approach.

          Fuad_YK 1 Reply Last reply Reply Quote 1
          • Fuad_YK
            Fuad_YK @ToughTimesLawyer last edited by

            Thanks for your view David, its much appreciated. Thanks, Fuad

            1 Reply Last reply Reply Quote 0
            • Fuad_YK
              Fuad_YK @RyanKent last edited by

              Thanks Ryan.

              Are the un-wanted/irrelevant pages likely to affect my organic seo?

              RyanKent Fuad_YK 2 Replies Last reply Reply Quote 0
              • RyanKent
                RyanKent @Fuad_YK last edited by

                It's impossible to say conclusively without examining your site and the content; however, since you refer to them as "junk" pages, it is likely they should best be removed to protect your other pages.

                1 Reply Last reply Reply Quote 0
                • Fuad_YK
                  Fuad_YK @Fuad_YK last edited by

                  Thanks Ryan.

                  1 Reply Last reply Reply Quote 0
                  • 1 / 1
                  • First post
                    Last post
                  • Why google stop index my home page
                    Andy.Drinkwater
                    Andy.Drinkwater
                    0
                    2
                    67

                  • Google Adding / Manipulating Page Meta Titles?
                    Dr-Pete
                    Dr-Pete
                    0
                    4
                    202

                  • Pages fluctuating +/- 70 positions in Google SERPs?
                    sbrault74
                    sbrault74
                    0
                    3
                    130

                  • Site not indexed on Google UK after 4 days?
                    digitalsoda
                    digitalsoda
                    0
                    3
                    109

                  • Stop google indexing CDN pages
                    loopyal
                    loopyal
                    0
                    3
                    2.7k

                  • Test contet/pages indexed by search engines
                    RyanKent
                    RyanKent
                    0
                    4
                    520

                  • Does Google index Wordpress pages with frames
                    BradBorst
                    BradBorst
                    0
                    3
                    922

                  • Why would my product pages no longer be indexed in Google?
                    KeriMorgret
                    KeriMorgret
                    0
                    9
                    2.7k

                  Get started with Moz Pro!

                  Unlock the power of advanced SEO tools and data-driven insights.

                  Start my free trial
                  Products
                  • Moz Pro
                  • Moz Local
                  • Moz API
                  • Moz Data
                  • STAT
                  • Product Updates
                  Moz Solutions
                  • SMB Solutions
                  • Agency Solutions
                  • Enterprise Solutions
                  • Digital Marketers
                  Free SEO Tools
                  • Domain Authority Checker
                  • Link Explorer
                  • Keyword Explorer
                  • Competitive Research
                  • Brand Authority Checker
                  • Local Citation Checker
                  • MozBar Extension
                  • MozCast
                  Resources
                  • Blog
                  • SEO Learning Center
                  • Help Hub
                  • Beginner's Guide to SEO
                  • How-to Guides
                  • Moz Academy
                  • API Docs
                  About Moz
                  • About
                  • Team
                  • Careers
                  • Contact
                  Why Moz
                  • Case Studies
                  • Testimonials
                  Get Involved
                  • Become an Affiliate
                  • MozCon
                  • Webinars
                  • Practical Marketer Series
                  • MozPod
                  Connect with us

                  Contact the Help team

                  Join our newsletter
                  Moz logo
                  © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                  • Accessibility
                  • Terms of Use
                  • Privacy