The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Robots.txt was set to disallow for 14 days

    Robots.txt was set to disallow for 14 days

    Intermediate & Advanced SEO
    3 2 58
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jc4254
      jc4254 last edited by

      We updated our website and accidentally  overwrote our  robots file with a version that prevented crawling  ( "Disallow: /") We realized the issue 14 days later and replaced  after our organic visits began to drop significantly and we quickly replace the robots file with the correct version to begin crawling again.  With  the impact to our organic visits, we have a few and any help would be greatly appreciated -

      Will the site get back to its original status/ranking ?

      If so .. how long would that take?

      Is there anything we can do to speed up the process ?

      Thanks

      1 Reply Last reply Reply Quote 0
      • seoelevated
        seoelevated last edited by

        This is commonly done intentionally, when launching a site on a new domain. Once the disallow is removed, the general practice is to request reindexing of the root domain page (and possibly some key pages with paths not likely to be found through navigation) in GSC, and also submitting (or re-submitting) your sitemaps directly in GSC (even though they also may/should be in your robots.txt file).

        I'm not sure how long you can expect the search engines to take, since your situation is a bit unique where the site was indexed, and then disallowed temporarily. Just guessing based on launching brand new domains, the process should be quick to be indexed (perhaps a few days) but might be slower on regaining previous ranking positions (unsure on timing of this).

        1 Reply Last reply Reply Quote 1
        • jc4254
          jc4254 last edited by

          Thank you for the response.

          We have been watching over the past week and there has been a very small change in the number of indexed urls in  GSC and no change in the stats on the  MOZ dashboard.

          Is that normal? How often does MOZ update the stats?

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Robots.txt Disallowed Pages and Still Indexed
            Igor.Go
            Igor.Go
            0
            3
            2.9k

          • Best practice for disallowing URLS with Robots.txt
            TimHolmes
            TimHolmes
            0
            3
            650

          • Should I disallow all URL query strings/parameters in Robots.txt?
            OlegKorneitchouk
            OlegKorneitchouk
            0
            5
            11.2k

          • Should I use meta noindex and robots.txt disallow?
            ntcma
            ntcma
            0
            5
            923

          • Should comments and feeds be disallowed in robots.txt?
            FedeEinhorn
            FedeEinhorn
            0
            5
            4.2k

          • Will disallowing in robots.txt noindex a page?
            FranckNlemba
            FranckNlemba
            0
            6
            510

          • Do I need to disallow the dynamic pages in robots.txt?
            esiow2013
            esiow2013
            0
            11
            1.0k

          • Robots.txt disallow subdomain
            oznappies
            oznappies
            0
            7
            1.9k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy