The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. One server, two domains - robots.txt allow for one domain but not other?

    One server, two domains - robots.txt allow for one domain but not other?

    Technical SEO Issues
    3 3 551
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Dave100
      Dave100 last edited by

      Hello,

      I would like to create a single server with two domains pointing to it. Ex: domain1.com -> myserver.com/     domain2.com -> myserver.com/subfolder. The goal is to create two separate sites on one server.

      I would like the second domain ( /subfolder) to be fully indexed / SEO friendly and have the robots txt file allow search bots to crawl. However, the first domain (server root) I would like to keep non-indexed, and the robots.txt file disallowing any bots / indexing.

      Does anyone have any suggestions for the best way to tackle this one?

      Thanks!

      1 Reply Last reply Reply Quote 0
      • SEO_Mechanic
        SEO_Mechanic last edited by

        A Records

        1 Reply Last reply Reply Quote 0
        • LesleyPaone
          LesleyPaone last edited by

          I am assuming that you are using Cpanel on the host for the site. Just add the second domain as an addon domain. Because of the way that things are set up in Cpanel and the way it handles redirects, the crawlers will not realize the local directory structure that one site is sitting in the sub folder of another site. They will be viewed as if they are two different sites on different servers. So you would handle the robtos.txt like they were on two different servers and just disregard anything about one being in the sub folder of another.

          1 Reply Last reply Reply Quote 2
          • 1 / 1
          • First post
            Last post
          • Multiple robots.txt files on server
            peakdistrictseo
            peakdistrictseo
            0
            8
            2.4k

          • Robots.txt blocking Addon Domains
            eglove
            eglove
            0
            10
            556

          • Robots.txt | any SEO advantage to having one vs not having one?
            JaneCopland
            JaneCopland
            0
            4
            119

          • I've consolidated other domains to a single one with 301 redirects, yet the new domain authority in MOZ is much less that the redirected ones. Is that right?
            MashBonigala
            MashBonigala
            2
            3
            232

          • Merge two different domains into one
            Bulpen
            Bulpen
            0
            3
            337

          • Is having no robots.txt file the same as having one and allowing all agents?
            ITRIX
            ITRIX
            0
            2
            697

          • Two Sites - One Domain
            ThompsonPaul
            ThompsonPaul
            0
            3
            217

          • Robots.txt and robots meta
            TheEspresseo
            TheEspresseo
            0
            5
            1.1k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy