The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. How to Disallow Specific Folders and Sub Folders for Crawling?

    How to Disallow Specific Folders and Sub Folders for Crawling?

    Intermediate & Advanced SEO
    14 3 1.5k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • CommercePundit
      CommercePundit @RyanKent last edited by

      Yes, I checked article to know more about it.

      http://httpd.apache.org/docs/1.3/howto/htaccess.html

      But, I am not able to find my solution. Can you suggest me specific article which suppose to help me more in same direction?

      RyanKent CommercePundit 3 Replies Last reply Reply Quote 0
      • vishalkhialani
        vishalkhialani last edited by

        also you can rename it so when google 's index shows up the results you won't get any hits.

        if thats what you want.

        CommercePundit 1 Reply Last reply Reply Quote -2
        • CommercePundit
          CommercePundit @vishalkhialani last edited by

          I can't getting you.

          vishalkhialani 1 Reply Last reply Reply Quote 0
          • vishalkhialani
            vishalkhialani @CommercePundit last edited by

            In addition to what Ryan mentioned I would rename that folder on your server. That will make google's index outdated and you won't get any visitors on the server

            1 Reply Last reply Reply Quote 0
            • RyanKent
              RyanKent @CommercePundit last edited by

              The link you shared is perfect. Near the top there is a link for OPTIONS. Click on it and you will be on this page: http://httpd.apache.org/docs/1.3/mod/core.html#options

              I want to very clearly state you should not make changes to your .htaccess file unless you are comfortable working with code. The slightest mistake and your entire site becomes unavailable. You can also damage the security of your site.

              With that said, if you decide to proceed anyway you can add the text I shared to the top of your .htaccess file. You definitely should BACK UP the file before making any changes.

              The suggestion vishalkialani made was to rename your /images folder to something else, perhaps /image. The problem is that if your site was not dynamically coded, you would break your image links.

              1 Reply Last reply Reply Quote 1
              • CommercePundit
                CommercePundit @CommercePundit last edited by

                Yes, I have similar problem with my code structure. Yesterday, I have set Relative path for all URLs. But, I am not sure about replacing of image name in code after make change in folder.

                So, I don't want to go with that manner. I also discussed with my development team and recommend to go with htaccess method.

                But, give me caution to follow specific method otherwise it may create big issue for crawling or indexing. 🙂 Right??

                1 Reply Last reply Reply Quote 0
                • RyanKent
                  RyanKent @CommercePundit last edited by

                  If you have a development team, they should instantly understand the problem.

                  A simple e-mail to any developer

                  E-mail title: Please fix

                  http://www.lampslightingandmore.com/images/

                  That's it. No other text should be needed. A developer should be able to look at the page and understand the index was left open and how to fix it. If you wish to be nicer then a simple "my index is open for the world to see, please don't allow public access to my server folders" should suffice.

                  1 Reply Last reply Reply Quote 2
                  • CommercePundit
                    CommercePundit last edited by

                    I am going to add following code to my htaccess page.

                    Options -Indexes

                    Will it work for me or not?

                    1 Reply Last reply Reply Quote 0
                    • CommercePundit
                      CommercePundit last edited by

                      I have added Options -Indexes for images folder in htaccess file.

                      But, I still able to find out images folder in Google indexing.

                      Can I check? Is it working properly or not? I don't want to index or display images folder in web search any more.

                      RyanKent 1 Reply Last reply Reply Quote 0
                      • RyanKent
                        RyanKent @CommercePundit last edited by

                        I checked your URL: http://www.lampslightingandmore.com/images/. The folder is now properly restricted and the images can no longer be seen using this method. Going forward, Google will not be able to index new images in the same manner your other images were indexed.

                        With respect to the images which have been indexed, I am not certain how Google will respond. The image links are still valid so they may keep them. On the other hand, the links are gone so they may remove them. If it were my site, I would wait 30 days to see if Google removed the results.

                        Another way you can resolve the issue is to change the file path to your images from /images to /image. This will immediately break all the links. You would need to ensure all the links on your site are updated properly. It still may take Google a month to de-index those results but it would certainly happen in that case.

                        CommercePundit RyanKent 2 Replies Last reply Reply Quote 1
                        • CommercePundit
                          CommercePundit @RyanKent last edited by

                          Can I use Remove URL facility from Google webmaster tools?

                          1 Reply Last reply Reply Quote 0
                          • RyanKent
                            RyanKent @RyanKent last edited by

                            You can, but then the content will be removed from Google's index for 90 days. I am not sure what effect this would have on pages with the images. It shouldn't have any effect, but I would hate for you to have rankings in any way affected for 90 days.

                            I have no experience in having images indexed in this manner. Perhaps someone else has more knowledge to share on this topic.

                            1 Reply Last reply Reply Quote 1
                            • 1 / 1
                            • First post
                              Last post
                            • Sub Directories Domain & Page Crawl Depth
                              effectdigital
                              effectdigital
                              0
                              2
                              53

                            • Google treats pages from main website and sub folder/sub directory differently?
                              Dalessi
                              Dalessi
                              0
                              3
                              552

                            • We 410'ed URLs to decrease URLs submitted and increase crawl rate, but dynamically generated sub URLs from pagination are showing as 404s. Should we 410 these sub URLs?
                              jeffchen
                              jeffchen
                              0
                              3
                              189

                            • What are the best practices for geo-targeting by sub-folders?
                              rjonesx. 0
                              rjonesx. 0
                              0
                              2
                              907

                            • Sub-Domain or Folder. Which is better for SEO.?
                              Keszi
                              Keszi
                              0
                              2
                              94

                            • Sub domain vs. sub folder
                              Linda-Vassily
                              Linda-Vassily
                              0
                              2
                              163

                            • E-commerce worldwide sub domains or folders
                              Letty
                              Letty
                              0
                              6
                              177

                            • Menu & Sub menu structure - section specific sub menu?
                              Ryan-Bradley
                              Ryan-Bradley
                              0
                              2
                              178

                            Get started with Moz Pro!

                            Unlock the power of advanced SEO tools and data-driven insights.

                            Start my free trial
                            Products
                            • Moz Pro
                            • Moz Local
                            • Moz API
                            • Moz Data
                            • STAT
                            • Product Updates
                            Moz Solutions
                            • SMB Solutions
                            • Agency Solutions
                            • Enterprise Solutions
                            • Digital Marketers
                            Free SEO Tools
                            • Domain Authority Checker
                            • Link Explorer
                            • Keyword Explorer
                            • Competitive Research
                            • Brand Authority Checker
                            • Local Citation Checker
                            • MozBar Extension
                            • MozCast
                            Resources
                            • Blog
                            • SEO Learning Center
                            • Help Hub
                            • Beginner's Guide to SEO
                            • How-to Guides
                            • Moz Academy
                            • API Docs
                            About Moz
                            • About
                            • Team
                            • Careers
                            • Contact
                            Why Moz
                            • Case Studies
                            • Testimonials
                            Get Involved
                            • Become an Affiliate
                            • MozCon
                            • Webinars
                            • Practical Marketer Series
                            • MozPod
                            Connect with us

                            Contact the Help team

                            Join our newsletter
                            Moz logo
                            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                            • Accessibility
                            • Terms of Use
                            • Privacy