The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. On-Page / Site Optimization
    4. Can Robots.txt on Root Domain override a Robots.txt on a Sub Domain?

    Can Robots.txt on Root Domain override a Robots.txt on a Sub Domain?

    On-Page / Site Optimization
    3 3 522
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • davelane.verve
      davelane.verve last edited by

      We currently have beta sites on sub-domains of our own domain. We have had issues where people forget to change the Robots.txt and these non-relevant beta sites get indexed by search engines (nightmare).

      We are going to move all of these beta sites to a new domain that we disallow all in the root of the domain.

      If we put fully configured Robots.txt on these sub-domains (that are ready to go live and open for crawling by the search engines) is there a way for the Robots.txt in the root domain to override the Robots.txt in these sub-domains?

      Apologies if this is unclear. I know we can handle this relatively easy by changing the Robots.txt in the sub-domain on going live but due to a few instances where people have forgotten I want to reduce the chance of human error!

      Cheers,

      Dave.

      1 Reply Last reply Reply Quote 0
      • dynamyt100
        dynamyt100 last edited by

        Dave, I had exactly the same issue a month ago with being indexed on subdomains but was able to modify robots.txt in root domain swiftly enough to avoid real damage. Your main root robots.txt can override the subdomains. Simply disallow the subdomain in your robots file disallow: /subdomain url/robots.txt OR, if im not mistaken simply remove it altogether from the subdomain

        1 Reply Last reply Reply Quote 1
        • RyanPurkey
          RyanPurkey last edited by

          Hi Dave. A workflow checklist should really help with this as well. There are probably a few other items you'll catch by meeting with the others involved and getting everyone on the same page. Cheers!

          1 Reply Last reply Reply Quote 1
          • 1 / 1
          • First post
            Last post
          • Correct robots.txt for WordPress
            jasongmcmahon
            jasongmcmahon
            1
            5
            180

          • Can I robots.txt an entire site to get rid of Duplicate content?
            Andy.Drinkwater
            Andy.Drinkwater
            0
            6
            139

          • Can you canonical from one domain page to a different domain page
            BeanstalkIM
            BeanstalkIM
            0
            8
            94

          • How can i make my sub-domain bring in more traffic?
            MichaelYork
            MichaelYork
            0
            2
            104

          • Campaign set up as sub domain
            PhilSmith23
            PhilSmith23
            0
            4
            164

          • Can I exclude sub-domains from the crawl diagnostics?
            JustinTaylor88
            JustinTaylor88
            0
            2
            443

          • Disallow a spammed sub-page from robots.txt
            zigojacko
            zigojacko
            0
            3
            475

          • New Domain or Sub Domain for Mobile content
            theshortstack
            theshortstack
            0
            4
            518

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy