The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Site being indexed by Google before it has launched

    Site being indexed by Google before it has launched

    Technical SEO Issues
    3 3 246
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Sayers
      Sayers last edited by

      We are currently coming towards the end of a site migration, and are at the final stage of testing redirects etc. However, to our horror we've just discovered Google has started indexing the new site. Any ideas on how this could have happened? I have most recently asked for robots.txt to exclude anything with a certain parameter in URL. Is there a chance this, wrongly implemented, could have caused this?

      1 Reply Last reply Reply Quote 0
      • HiveDigitalInc
        HiveDigitalInc last edited by

        Many ways - Google discovers URLs through a large number of methods, although primarily through links. I have seen some pretty amazing ways of discovery though...

        • Links posted in emails where the emails ended up on the web (like a private newsletter with a public archive)
        • Links showing up in click stream data services like alexa
        • Links showing up from "recently registered" domain lists

        The rule of thumb is always ALWAYS start with a robots.txt. It is the first thing you should do when setting up a dev environment.

        1 Reply Last reply Reply Quote 0
        • KeriMorgret
          KeriMorgret last edited by

          Duplicate question, closing this question so all answers can be given at http://www.seomoz.org/q/site-being-indexed-by-google-before-it-has-launched-2

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Removing a site from Google index with no index met tags
            effectdigital
            effectdigital
            1
            5
            58

          • Should you use robots.txt for pages within your site which do not have high quality content or are not contributing a great deal so when Google crawls your site the best performing content has a higher chance of being indexed?
            Jacksons_Fencing
            Jacksons_Fencing
            0
            5
            44

          • Anything new if determining how many of a sites pages are in Google's supplemental index vs the main index?
            SEMPassion
            SEMPassion
            0
            4
            390

          • Staging site and "live" site have both been indexed by Google
            jesse-landry
            jesse-landry
            0
            6
            1.3k

          • Why did Google stop indexing my site?
            JillB2013
            JillB2013
            0
            12
            5.3k

          • Why my site is not indexing in google
            Rajesh.Chandran
            Rajesh.Chandran
            0
            5
            216

          • A site is not being indexed by Google Yahoo or Bing
            Saijo.George
            Saijo.George
            0
            6
            539

          • Removing a site from Google's index
            issuebasedmedia
            issuebasedmedia
            0
            13
            976

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy