The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Ignore Urls with pattern.

    Ignore Urls with pattern.

    Technical SEO Issues
    5 3 591
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • levalencia1
      levalencia1 last edited by

      I have 7000 warnings of urls because of a 302 redirect.

      http://imageshack.us/photo/my-images/215/44060409.png/

      I want to get rid of those, is it possible to get rid of the Urls with robots.txt.

      For example that it does not crawl anything that has /product_compare/ in its url?

      Thank you

      1 Reply Last reply Reply Quote 0
      • rasmusbang
        rasmusbang last edited by

        Could you perhaps post a URL which has product_compare in it?

        You could alter your robots.txt file to disallow robots to index pages in

        http//www.domain.com/product_compare/ by adding this line to your robots.txt file: Disallow: /product_compare/

        levalencia1 1 Reply Last reply Reply Quote 0
        • levalencia1
          levalencia1 @rasmusbang last edited by

          http://www.theprinterdepo.com/catalog/product_compare/add/product/100/uenc/aHR0cHM6Ly93d3cudGhlcHJpbnRlcmRlcG8uY29tL2hwLW1haW50ZW5hY2Uta2l0LWZvci00LWxqNC1sajUtZm9yLWV4Y2hhbmdlLXJlZnViaXNoZWQ,/

          In the image you can see  many also 🙂

          rasmusbang AlanMosley 2 Replies Last reply Reply Quote 0
          • rasmusbang
            rasmusbang @levalencia1 last edited by

            Then you simply add this to your robots.txt:

            Disallow: /catalog/product_compare/

            That should leave out all pages starting with:
            https://www.theprinterdepo.com/catalog/product_compare/

            1 Reply Last reply Reply Quote 0
            • AlanMosley
              AlanMosley @levalencia1 last edited by

              in case they do not all start with /category

              Disallow: product_compare

              1 Reply Last reply Reply Quote 1
              • 1 / 1
              • First post
                Last post
              • Adding a parameter to the URL / URL Stracture
                Roman-Delcarmen
                Roman-Delcarmen
                0
                2
                58

              • WP URL issue - Concatenated URLs (LOTS of them)
                Nigel_Carr
                Nigel_Carr
                0
                2
                32

              • How can I make it so that robots.txt is not ignored due to a URL re-direct?
                rodelmo4
                rodelmo4
                0
                4
                56

              • Absolute URL or Relative URL in my sitemap?
                brianvest
                brianvest
                0
                3
                9.6k

              • Why is google webmaster tools ignoring my url parameter settings
                mark_baird
                mark_baird
                0
                3
                290

              • Friendly URLS (SEO urls)
                irvingw
                irvingw
                0
                5
                217

              • Updating content on URL or new URL
                RuudHeijnen
                RuudHeijnen
                0
                7
                273

              • How to find original URLS after Hosting Company added canonical URLs, URL rewrites and duplicate content.
                Nobody1560986989723
                Nobody1560986989723
                0
                2
                366

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy