The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Effect duration of robots.txt file.

    Effect duration of robots.txt file.

    Intermediate & Advanced SEO
    4 4 255
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • innofidelity
      innofidelity last edited by

      in my web site there is demo site in that also, index in Google but no need it now.so i have created robots file and upload to server yesterday.in the demo folder there are some html files,and i wanna remove all these in demo file from Google.but still in web master tools it showing

      User-agent: *
      Disallow: /demo/

      How long this will take to remove from Google ?

      And are there any alternative way doing that ?

      1 Reply Last reply Reply Quote 0
      • AJPro
        AJPro last edited by

        Hello,

        The robots directive will only prevent google from crawling the pages. In order t remove the pages from index you need to add "meta noindex" to the pages you want to have removed.

        <meta name="robots" content="noindex">
        

        http://support.google.com/webmasters/bin/answer.py?hl=en&answer=93710

        josh-riley 1 Reply Last reply Reply Quote 2
        • josh-riley
          josh-riley @AJPro last edited by

          And, if they are already indexed, you have to wait for them to be recrawled, then fall out of index, so it's not an immediate thing. Sometimes it takes days, sometimes weeks.

          1 Reply Last reply Reply Quote 1
          • KeriMorgret
            KeriMorgret last edited by

            Google Webmaster Tools also has a remove URL function where you can remove an entire directory, which may be of help to you.

            1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post
            • Will disallowing URL's in the robots.txt file stop those URL's being indexed by Google
              Martijn_Scheijbeler
              Martijn_Scheijbeler
              0
              11
              1.6k

            • Meta robots or robot.txt file?
              Andy.Drinkwater
              Andy.Drinkwater
              0
              5
              152

            • What should I block with a robots.txt file?
              Travis-W
              Travis-W
              1
              3
              298

            • Files blocked in robot.txt and seo
              john4math
              john4math
              0
              4
              344

            • Could you use a robots.txt file to disalow a duplicate content page from being crawled?
              KyleChamp
              KyleChamp
              0
              11
              1.3k

            • Can I use a "no index, follow" command in a robot.txt file for a certain parameter on a domain?
              NakulGoyal
              NakulGoyal
              0
              3
              460

            • Negative impact on crawling after upload robots.txt file on HTTPS pages
              ShaMenz
              ShaMenz
              0
              2
              892

            • Why is noindex more effective than robots.txt?
              webseoservices
              webseoservices
              0
              3
              705

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy