The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Web Design
    4. Block parent folder in robot.txt, but not children

    Block parent folder in robot.txt, but not children

    Web Design
    3 3 547
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Zuken
      Zuken last edited by

      Example:

      I want to block this URL (which shows up in Webmaster Tools as an error):

      http://www.siteurl.com/news/events-calendar/usa

      But not this:

      http://www.siteurl.com/news/events-calendar/usa/event-name

      1 Reply Last reply Reply Quote 0
      • ABullis
        ABullis last edited by

        You could use "allow" in your robots.txt file for just this problem.

        allow: news/events-calendar/usa/event-name

        disallow: /news/events-calendar/usa

        See the allow directive section of this page: https://en.wikipedia.org/wiki/Robots_exclusion_standard

        1 Reply Last reply Reply Quote 1
        • Martijn_Scheijbeler
          Martijn_Scheijbeler last edited by

          The idea from Andrew is nice, but my guess would be that you're targeting multiple events so that might run into issues. What you could do is add some more regular expression and make it like this:

          Disallow: ^/news/events-calendar/usa$

          1 Reply Last reply Reply Quote 1
          • 1 / 1
          • First post
            Last post
          • Google Search Console Block
            BoostMyBiz
            BoostMyBiz
            0
            7
            125

          • Robots.txt being blocked
            SOM24
            SOM24
            0
            9
            1.4k

          • How to command Robots.txt to this:
            Chenzo
            Chenzo
            0
            7
            151

          • Not able to serve different robots.txt for CDN77
            Aman_123
            Aman_123
            0
            3
            282

          • How to rewrite/redirect a folder name with .htaccess
            MickEdwards
            MickEdwards
            0
            6
            26.3k

          • Robots.txt - Allow and Disallow. Can they be the same?
            eventurerob
            eventurerob
            0
            9
            801

          • How to fix and issue with robot.txt ?
            Matt-Williamson
            Matt-Williamson
            0
            7
            4.0k

          • Correct use for Robots.txt
            EricVallee34
            EricVallee34
            0
            5
            1.0k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy