The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Robots.txt on subdomains

    Robots.txt on subdomains

    Technical SEO Issues
    3 2 3.5k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Whittie
      Whittie last edited by

      Hi guys!

      I keep reading conflicting information on this and it's left me a little unsure. Am I right in thinking that a website with a subdomain of shop.sitetitle.com will share the same robots.txt file as the root domain?

      1 Reply Last reply Reply Quote 0
      • RyanPurkey
        RyanPurkey last edited by

        Mostly no.  I say 'mostly' because a lot of times when you look at a site using www and no-www if both of those work they're almost always pulling files from the same location (hence the warnings around duplicate content), so both www.domain.com/robots.txt and domain.com/robots.txt are going to work.  This is the dominant example of a subdomain sharing a robots.txt file. However, on domains that are set up as their own subdomains they have different robots.txt.  Take a look at the many differences between subdomain1-1000.wordpress.com/robots.txt vs wordpress.com/robots.txt.  If you set up a subdomain that isn't just a reflection of your root domain, then you'll need to create a robots.txt file as well. Cheers!

        Whittie 1 Reply Last reply Reply Quote 4
        • Whittie
          Whittie @RyanPurkey last edited by

          That's about as comprehensive an answer as I could have hoped for. Thanks Ryan, really appreciated.

          1 Reply Last reply Reply Quote 1
          • 1 / 1
          • First post
            Last post
          • Robots.txt
            MichaelC-15022
            MichaelC-15022
            0
            7
            1.0k

          • Robots.txt
            Dan-Lawrence
            Dan-Lawrence
            0
            5
            99

          • Robots.txt
            WesleySmits
            WesleySmits
            0
            9
            168

          • Do I need robots.txt and meta robots?
            Cyrus-Shepard
            Cyrus-Shepard
            0
            7
            1.1k

          • Subdomain Removal in Robots.txt with Conditional Logic??
            KeriMorgret
            KeriMorgret
            0
            3
            1.9k

          • Robots.txt and robots meta
            TheEspresseo
            TheEspresseo
            0
            5
            1.1k

          • Robots.txt
            Tom-Anthony
            Tom-Anthony
            0
            4
            1.1k

          • Subdomain Robots.txt
            JohnECF
            JohnECF
            0
            3
            1.2k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy