The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Exclude root url in robots.txt ?

    Exclude root url in robots.txt ?

    Technical SEO Issues
    5 2 122
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • mikehenze
      mikehenze last edited by

      Hi,

      I have the following setup:

      www.example.com/nl
      www.example.com/de
      www.example.com/uk
      etc
      www.example.com is 301'ed to www.example.com/nl

      But now www.example.com is ranking instead of www.example.com/nl
      Should is block www.example.com in robots.txt so only the subfolders are being ranked?
      Or will i lose my ranking by doing this.

      1 Reply Last reply Reply Quote 0
      • Philip-DiPatrizio
        Philip-DiPatrizio last edited by

        If you put disallow: / in your robots.txt file, you will tell bots not to crawl the homepage plus ALL interior pages.  You'd be shooting yourself in the foot (or head, really).

        Are you sure the redirect is setup properly?  Is it definitely a 301 redirect, or maybe a 302 (temporary)?  How long ago did you implement the redirect?  If the 301 redirect is setup properly, and you're still seeing the homepage in the index, you might just need to wait for it to drop out.

        mikehenze 1 Reply Last reply Reply Quote 1
        • mikehenze
          mikehenze @Philip-DiPatrizio last edited by

          Thanks for answering Philip,

          Yes i really used a 301.
          I used it in .htaccess
          And i have set this up before lauching the site, so i should be good from the beginning.

          The site was launched last friday.
          When searching for the brand name it shows up as example.com
          When searching for my main keywords it shows example.com/ned/landing-page

          Philip-DiPatrizio 1 Reply Last reply Reply Quote 0
          • Philip-DiPatrizio
            Philip-DiPatrizio @mikehenze last edited by

            The site just launched?  It sounds like I am right, you just need to give Google some time to drop the page from the index.

            When you find the homepage in the index, and you click the link, do you get redirected?  If so, Google will eventually drop it.

            mikehenze 1 Reply Last reply Reply Quote 1
            • mikehenze
              mikehenze @Philip-DiPatrizio last edited by

              Yes, when clicking the link in google you get redirected.
              I will wait some time.

              Thank you.

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • "Url blocked by robots.txt." on my Video Sitemap
                sergeystefoglo
                sergeystefoglo
                0
                3
                521

              • What are the negative implications of listing URLs in a sitemap that are then blocked in the robots.txt?
                Dezzign
                Dezzign
                0
                3
                153

              • Why is robots.txt blocking URL's in sitemap?
                PurpleGriffon
                PurpleGriffon
                0
                3
                346

              • Question about construction of our sitemap URL in robots.txt file
                danatanseo
                danatanseo
                0
                16
                1.6k

              • No indexing url including query string with Robots txt
                HMK-NL
                HMK-NL
                0
                7
                19.2k

              • Can I Disallow Faceted Nav URLs - Robots.txt
                AlanMosley
                AlanMosley
                0
                5
                914

              • Robots.txt and robots meta
                TheEspresseo
                TheEspresseo
                0
                5
                1.1k

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy