The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Search Engine Trends
    4. Adding non-important folders to disallow in robots.txt file

    Adding non-important folders to disallow in robots.txt file

    Search Engine Trends
    4 3 162
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • vtmoz
      vtmoz last edited by

      Hi all,

      If we have many non-important folders like /category/ in blog.....these will multiply the links. These are strictly for users who access very rarely but not for bots. Can we add such to disallow list in robots to stop link juice passing from them, so internal linking will me minimised to an extent. Can we add any such paths or pages in disallow list? Is this going to work pure technical or any penalty?

      Thanks,

      Satish

      1 Reply Last reply Reply Quote 0
      • Alick300
        Alick300 last edited by

        Hi,

        Yes you can block in robots.txt. You can also use rel="nofollow" if you don't want to pass link juice.

        [No Link Juice](https://www.example.com)
        
        Hope this helps.
        Thanks
        
        vtmoz 1 Reply Last reply Reply Quote 0
        • Hurf
          Hurf last edited by

          This is a great resource for all things robots.txt related:
          
          [http://www.robotstxt.org/robotstxt.html](http://www.robotstxt.org/robotstxt.html)
          
          1 Reply Last reply Reply Quote 0
          • vtmoz
            vtmoz @Alick300 last edited by

            But as per the current SEO buzz,  internal nofollow leads of waste of link juice and we cannot preserve it. Moreover some suggests not to use nofollow internally.

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Folders or no folders in url?
              MrWhippy
              MrWhippy
              0
              10
              2.6k

            • Need only tens of pages to be indexed out of hundreds: Robots.txt is Okay for Google to proceed with?
              GastonRiera
              GastonRiera
              0
              2
              24

            • Have you ever seen or experienced a page indexed which is actually from a website which is blocked by robots.txt?
              GastonRiera
              GastonRiera
              0
              2
              54

            • Meta robots at every page rather than using robots.txt for blocking crawlers? How they'll get indexed if we block crawlers?
              ThompsonPaul
              ThompsonPaul
              0
              3
              253

            • The importance of meta keywords?
              roundbrix
              roundbrix
              0
              10
              10.0k

            • Adding the link masking directory to robots.txt?
              Everett
              Everett
              0
              3
              243

            • URL Importance In Search
              Boodreaux
              Boodreaux
              0
              6
              233

            • KML File vs. KMZ File
              kadesmith
              kadesmith
              0
              4
              946

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy