The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. How long after disallowing Googlebot from crawling a domain until those pages drop out of their index?

    How long after disallowing Googlebot from crawling a domain until those pages drop out of their index?

    Technical SEO Issues
    3 3 99
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • TLM
      TLM last edited by

      We recently had Google crawl a version of the site we that we had thought we had disallowed already. We have corrected the issue of them crawling the site, but pages from that version are still appearing in the search results (the version we want them to not index and serve up is our .us domain which should have been blocked to them).

      My question is this: How long should I expect that domain (the .us we don't want to appear) to stay in their index after disallowing their bot? Is this a matter of days, weeks, or months?

      1 Reply Last reply Reply Quote 0
      • Shakur
        Shakur last edited by

        Hi there,

        Crawling and indexing are processes which can take some time and which rely on many factors. In general, we cannot make predictions or guarantees about when or if your URLs will be crawled or indexed.

        Hope it helps you.

        1 Reply Last reply Reply Quote 0
        • MickEdwards
          MickEdwards last edited by

          If it is the case that no URL for .us should exist (there are not new URLs) then you can remove pretty swiftly in Webmaster Tools >> Google Index >> Remove URLs >> select the root URL and select to remove all directories that come from it.

          1 Reply Last reply Reply Quote 1
          • 1 / 1
          • First post
            Last post
          • URLs dropping from index (Crawled, currently not indexed)
            DmitriiK
            DmitriiK
            0
            2
            38

          • My pages are being crawled, but not indexed according to Search Console
            DonnaDuncan
            DonnaDuncan
            0
            4
            64

          • Drop in traffic, spike in indexed pages
            rjonesx. 0
            rjonesx. 0
            0
            2
            145

          • What would cause a sudden drop in indexed sitemap pages?
            EcommerceSite
            EcommerceSite
            0
            4
            787

          • New pages need to be crawled & indexed
            SSFCU
            SSFCU
            0
            4
            84

          • Can Googlebot crawl the content on this page?
            danatanseo
            danatanseo
            0
            7
            156

          • Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
            Nobody1560986989723
            Nobody1560986989723
            0
            5
            422

          • Why googlebot indexing one page, not the other?
            john4math
            john4math
            0
            2
            535

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy