The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?

    If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?

    Intermediate & Advanced SEO
    3 3 98
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Gabriele_Layoutweb
      Gabriele_Layoutweb last edited by

      If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?

      1 Reply Last reply Reply Quote 0
      • Andy.Drinkwater
        Andy.Drinkwater last edited by

        Hi Jaro,

        Head into search console and use the Temporary Remove URL tool - this should work pretty quickly. The next time Google comes around to that page, they should see the NOINDEX flag and not re-index it.

        -Andy

        1 Reply Last reply Reply Quote 3
        • GastonRiera
          GastonRiera last edited by

          Hello Jaro,

          What Andy says is right, im backing him up. Remember to not include that URL in the sitemap.

          Also is a good moment to say that with the robots.txt you just tell google bot not no follow it, that differs from indexing it. There are cases where URLs are indexed instead of being "blocked" in the robots.txt.

          The fine way stop google from indexing a certain URL would be adding the meta robots tag including a noindex atribute.
          Here there a quote from the Webmaster central help forum in Google:

          If you block a file from crawling and Google discovers a URL for that file on another site, it may still index the file using whatever information it can find, even though crawling is blocked. So robots.txt disallow does not necessarily stop something being indexed.
          (in the ets answer, a note below the Disallow part)

          Hope it's clarifying.
          Best luck.
          GR.

          1 Reply Last reply Reply Quote 2
          • 1 / 1
          • First post
            Last post
          • Block session id URLs with robots.txt
            Mat_C
            Mat_C
            1
            4
            130

          • Is robots met tag a more reliable than robots.txt at preventing indexing by Google?
            Bobbi_Tschumper
            Bobbi_Tschumper
            1
            7
            3.0k

          • How to make Google index your site? (Blocked with robots.txt for a long time)
            SanjidaKazi
            SanjidaKazi
            0
            3
            128

          • Pages getting into Google Index, blocked by Robots.txt??
            Devanur-Rafi
            Devanur-Rafi
            0
            10
            673

          • How long does google index old urls?
            daracreative
            daracreative
            0
            3
            108

          • Blocking out specific URLs with robots.txt
            Modi
            Modi
            0
            3
            133

          • Google showing high volume of URLs blocked by robots.txt in in index-should we be concerned?
            TakeshiYoung
            TakeshiYoung
            0
            4
            302

          • Will blocking urls in robots.txt void out any backlink benefits? - I'll explain...
            AubieJon
            AubieJon
            0
            4
            1.0k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy