The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. What are your thoughts on security of placing CMS-related folders in a robots.txt file?

    What are your thoughts on security of placing CMS-related folders in a robots.txt file?

    Technical SEO Issues
    4 4 431
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • James-Distinction
      James-Distinction last edited by

      So I was just about to add a whole heap of CMS-related folders to my robots.txt file to exclude them from search, and thought "hey, I'm publicly telling people where my admin folders are"...surely that's not right?!

      Should I leave them out of the robots.txt file, and hope for the best that they never get indexed? Should I use noindex meta data on every page?

      What are people's thoughts?

      Thanks,

      James

      PS. I know this is similar to lots of other discussions around meta noindex vs. robots.txt, but I'm after specific thoughts around the security aspect of listing your admin folders in a robots.txt file...

      1 Reply Last reply Reply Quote 0
      • sesertin
        sesertin last edited by

        I found three options for you: http://www.techiecorner.com/106/how-to-disable-directory-browsing-using-htaccess-apache-web-server/

        I think if you do it with.htacces that is a folder specific file than nobody will be able to detect where admin contet is located.

        1 Reply Last reply Reply Quote 0
        • RyanKent
          RyanKent last edited by

          As a rule, you want to avoid using robots.txt files whenever possible. It does not consistently protect you from crawlers and when it does block crawlers it kills any PR on those pages.

          If you can block those pages with a noindex tag, it would be a preferable solution.

          With respect to security for a CMS site, it really needs to be a comprehensive effort. Many site owners take a couple steps and then have a false-sense of security. Here are a few thoughts:

          • try the site address with /administrator after it to access Joomla and other sites

          • try the site address or blog with /wp-admin/ after it to access Joomla sites

          • make up a webpage and try accessing it to view the site's 404 page

          • right-click on a page and choose View Page Source. Often you will see the name of the CMS clearly listed. Other times you will see clear clues such as /wp/ in folder names. Other times you will find unique extensions such as Yoast SEO which will give you an idea of the CMS

          Once a bad guy knows which CMS is in use, they know the default folder structure and more. The point is it requires a lot more effort then most people realize to hide the CMS in use. I applaud your effort, but be very thorough about it. There is a lot more involved then simply covering your robots.txt file.

          1 Reply Last reply Reply Quote 0
          • AlanMosley
            AlanMosley last edited by

            surly your admin folders are secured?, it would not matter if someone knows where they are.

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Is there any value in having a blank robots.txt file?
              LiamVictor
              LiamVictor
              0
              6
              3.1k

            • Meta Robots Noindex and Robots.txt File
              Devanur-Rafi
              Devanur-Rafi
              0
              2
              125

            • Does this robots.txt file look right?
              ThompsonPaul
              ThompsonPaul
              0
              8
              188

            • Do i have my robots.txt file set up properly
              ClaireH-184886
              ClaireH-184886
              1
              4
              319

            • Does Bing ignore robots txt files?
              Nightwing
              Nightwing
              0
              3
              2.8k

            • Robots.txt Sitemap with Relative Path
              ClickConsult
              ClickConsult
              0
              2
              8.1k

            • Use of Robots.txt file on a job site
              jennita
              jennita
              0
              5
              850

            • Restricted by robots.txt and soft bounce issues (related).
              RyanKent
              RyanKent
              0
              4
              772

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy