The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Robot.txt : How to block a specific file type in several subdirectories ?

    Robot.txt : How to block a specific file type in several subdirectories ?

    Technical SEO Issues
    3 2 1.0k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • LabeliumUSA
      LabeliumUSA last edited by

      Hello everyone !

      I need help setting up a robot.txt.

      I'm trying to block all pdf files in particular directories so I'm using this command. In the example below the line is blocking all .gif in the entire site.

      Block files of a specific file type (for example, .gif) | Disallow: /*.gif$

      2 questions :

      • Can I use this command to specify one particular directory in which I want to block pdf files ? Will this line be recognized by googlebots ?

      Disallow: /fileadmin/xxxxxxx/xxx/xxxxxxx/*.pdf$

      • Then I realized that I would have to write as many lines as many directories there are in which I want to block pdf files.

      Let's say I want to block pdf files in all these 3 directories

      /fileadmin/directory1

      /fileadmin/directory1/sub1

      /fileadmin/directory1/sub1/pdf

      Is there a pattern-matching rule I could use to blocks access to pdf files in all subdirectories instead of writing 3x the above line for each subdirectory ? For exemple :

      Disallow: /fileadmin/directory1*/

      Many thanks in advance for any insight you may have.

      1 Reply Last reply Reply Quote 0
      • Rajesh.Prajapati
        Rajesh.Prajapati last edited by

        Use this code -
        Disallow: /*.f$
        If you want to block only one folder then use this -
        Disallow: /folder1/
        .*f$
        This rule will help to block both files only .pdf and .gif

        LabeliumUSA 1 Reply Last reply Reply Quote 1
        • LabeliumUSA
          LabeliumUSA @Rajesh.Prajapati last edited by

          Hey thank you for your answer, really appreciate it.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Blocking subdomains with Robots.txt file
            PaulM01
            PaulM01
            0
            3
            641

          • Should I block Map pages with robots.txt?
            imaginex
            imaginex
            0
            3
            73

          • Meta Robots Noindex and Robots.txt File
            Devanur-Rafi
            Devanur-Rafi
            0
            2
            125

          • Does this robots.txt file look right?
            ThompsonPaul
            ThompsonPaul
            0
            8
            188

          • Do i have my robots.txt file set up properly
            ClaireH-184886
            ClaireH-184886
            1
            4
            319

          • Blocked by meta-robots but there is no robots file
            Twinbytes
            Twinbytes
            0
            9
            700

          • Robots.txt blocking site or not?
            RyanKent
            RyanKent
            0
            2
            445

          • Blocking other engines in robots.txt
            RyanKent
            RyanKent
            0
            2
            581

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy