The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Files blocked in robot.txt and seo

    Files blocked in robot.txt and seo

    Intermediate & Advanced SEO
    4 2 344
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • seoanalytics
      seoanalytics last edited by

      I use joomla and I have blocked the following in my robots.txt is there anything that is bad for seo ?

      User-agent: *

      Disallow: /administrator/

      Disallow: /cache/

      Disallow: /components/

      Disallow: /images/

      Disallow: /includes/

      Disallow: /installation/

      Disallow: /language/

      Disallow: /libraries/

      Disallow: /media/

      Disallow: /modules/

      Disallow: /plugins/

      Disallow: /templates/

      Disallow: /tmp/

      Disallow: /xmlrpc/

      Disallow: /mailto:myemail@myemail.com/

      Disallow: /javascript:void(0)

      Disallow: /.pdf

      1 Reply Last reply Reply Quote 0
      • john4math
        john4math last edited by

        If you don't want pages in those Disallowed directories to be indexed, then you're doing fine.  These pages won't be able to be crawled, so, they won't be likely to appear in search results for any search engines.

        The last three entries look fishy to me.  I'd need to know what types of URLs you're trying to block to fix them.  For the last one, if you're looking to block all pdfs on your site, the syntax would be Disallow: /*.pdf.

        seoanalytics 1 Reply Last reply Reply Quote 0
        • seoanalytics
          seoanalytics @john4math last edited by

          Thank you and is blocking javascript bad ? ( I was thinking about submenus )

          john4math 1 Reply Last reply Reply Quote 0
          • john4math
            john4math @seoanalytics last edited by

            What you have there is just blocking rootdomain.com/javascript:void(0).  Googlebot can execute and index JavaScript; you should not block it without a good reason.  I'd let it read the JavaScript and see the submenus.

            1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post
            • Robots.txt blocked internal resources Wordpress
              Mat_C
              Mat_C
              1
              5
              985

            • Large robots.txt file
              ThomasHarvey
              ThomasHarvey
              0
              2
              513

            • Meta robots or robot.txt file?
              Andy.Drinkwater
              Andy.Drinkwater
              0
              5
              152

            • Block subdomain directory in robots.txt
              DirkC
              DirkC
              0
              5
              1.1k

            • Blocking out specific URLs with robots.txt
              Modi
              Modi
              0
              3
              133

            • Issue with Robots.txt file blocking meta description
              ThompsonPaul
              ThompsonPaul
              0
              9
              3.8k

            • Is our robots.txt file correct?
              Igal_Zeifman
              Igal_Zeifman
              0
              5
              175

            • Blocking Dynamic URLs with Robots.txt
              TaitLarson
              TaitLarson
              1
              4
              5.1k

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy