The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Using 2 wildcards in the robots.txt file

    Using 2 wildcards in the robots.txt file

    Intermediate & Advanced SEO
    2 2 605
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • seo123456
      seo123456 last edited by

      I have a URL string which I don't want to be indexed. it includes the characters _Q1 ni the middle of the string.

      So in the robots.txt can I use 2 wildcards in the string to take out all of the URLs with that in it?  So something like /_Q1.  Will that pickup and block every  URL with those characters in the string?

      Also, this is not directly of the root, but in a secondary directory, so .com/.../_Q1.  So do I have to format the robots.txt as //_Q1* as it will be in the second folder or just using /_Q1 will pickup everything no matter what folder it is on?

      Thanks.

      1 Reply Last reply Reply Quote 0
      • lonniea
        lonniea last edited by

        I'm not 100% positive, however it does make sense to use it this way.

        User-agent: *
        Disallow: /*_Q1$

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Set Robots.txt file to crawl my website at specific times
          Tenlo
          Tenlo
          0
          2
          50

        • If my website do not have a robot.txt file, does it hurt my website ranking?
          Alick300
          Alick300
          0
          2
          242

        • Using Meta Header vs Robots.txt
          evan89
          evan89
          0
          16
          195

        • Robot.txt File Not Appearing, but seems to be working?
          Vizergy
          Vizergy
          0
          4
          91

        • What should I block with a robots.txt file?
          Travis-W
          Travis-W
          1
          3
          298

        • Could you use a robots.txt file to disalow a duplicate content page from being crawled?
          KyleChamp
          KyleChamp
          0
          11
          1.3k

        • Can I use a "no index, follow" command in a robot.txt file for a certain parameter on a domain?
          NakulGoyal
          NakulGoyal
          0
          3
          460

        • Block all but one URL in a directory using robots.txt?
          Cyrus-Shepard
          Cyrus-Shepard
          0
          3
          3.5k

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy