The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Disallow URLs ENDING with certain values in robots.txt?

    Disallow URLs ENDING with certain values in robots.txt?

    Intermediate & Advanced SEO
    4 2 1.9k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jmorehouse
      jmorehouse last edited by

      Is there any way to disallow URLs ending in a certain value? For example, if I have the following product page URL: http://website.com/category/product1, and I want to disallow /category/product1/review, /category/product2/review, etc. without disallowing the product pages themselves, is there any shortcut to do this, or must I disallow each gallery page individually?

      1 Reply Last reply Reply Quote 0
      • Andy.Drinkwater
        Andy.Drinkwater last edited by

        So you just want to disallow the /review/ element?

        *Disallow: /review/

        I am pretty sure the wildcard will work.

        I would use the Robots.txt tester in Webmaster tools to try it out first before committing to any changes. This will tell you if it works and if so, if it's blocked successfully.

        You then try the URL without the /review/ element on and make sure it passes.

        -Andy

        jmorehouse 1 Reply Last reply Reply Quote 1
        • jmorehouse
          jmorehouse @Andy.Drinkwater last edited by

          Thanks, Andy! I just tested with Webmaster tool's tester, and it worked.

          Andy.Drinkwater 1 Reply Last reply Reply Quote 1
          • Andy.Drinkwater
            Andy.Drinkwater @jmorehouse last edited by

            Excellent stuff. Glad it helped 🙂

            -Andy

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Block session id URLs with robots.txt
              Mat_C
              Mat_C
              1
              4
              130

            • If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?
              GastonRiera
              GastonRiera
              0
              3
              98

            • Will disallowing URL's in the robots.txt file stop those URL's being indexed by Google
              Martijn_Scheijbeler
              Martijn_Scheijbeler
              0
              11
              1.6k

            • Should I disallow all URL query strings/parameters in Robots.txt?
              OlegKorneitchouk
              OlegKorneitchouk
              0
              5
              11.2k

            • Blocking out specific URLs with robots.txt
              Modi
              Modi
              0
              3
              133

            • Robots.txt: Syntax URL to disallow
              Anti-Alex
              Anti-Alex
              0
              8
              479

            • If i disallow unfriendly URL via robots.txt, will its friendly counterpart still be indexed?
              Cyrus-Shepard
              Cyrus-Shepard
              0
              4
              620

            • How to Disallow Tag Pages With Robot.txt
              monster99
              monster99
              0
              6
              4.0k

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy