The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Should we block urls like this - domainname/shop/leather-chairs.html?brand=244&cat=16&dir=ascℴ=price&price=1 within the robots.txt?

    Should we block urls like this - domainname/shop/leather-chairs.html?brand=244&cat=16&dir=ascℴ=price&price=1 within the robots.txt?

    Intermediate & Advanced SEO
    2 2 864
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • MonsterWeb28
      MonsterWeb28 last edited by

      I've recently added a campaign within the SEOmoz interface and received an alarming number of errors ~9,000 on our eCommerce website. This site was built in Magento, and we are using search friendly url's however most of our errors were duplicate content / titles due to url's like: domainname/shop/leather-chairs.html?brand=244&cat=16&dir=asc&order=price&price=1 and domainname/shop/leather-chairs.html?brand=244&cat=16&dir=asc&order=price&price=4.

      Is this hurting us in the search engines? Is rogerbot too good?

      What can we do to cut off bots after the ".html?" ? Any help would be much appreciated 🙂

      1 Reply Last reply Reply Quote 0
      • sferrino
        sferrino last edited by

        I had the same problem on http://www.tokenrock.com because I was doing a lot of URL Rewriting, it's a CMS system I wrote, but the same issue apply. I went from 7000+ errors according to SEOMoz, and I'm down to 700. Here's a few things I did:

        Use canonicals on everything you possibly can.

        Redirect 301 the items in the SERPS that are identical.

        I'm not familiar with Magento to help you work though that side of it.

        Having a link like: domainname/leather-chairs-244-16-price-1.html would work much better.

        The ones you have listed are because somehow somewhere you (the site) have a link to it.

        Unfortunately some of the CMS's are written by developers who don't fully understand SEO and why the ? is a bad thing.

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Robots.txt & Disallow: /*? Question!
          BabaBha0173
          BabaBha0173
          0
          8
          321

        • If I block a URL via the robots.txt - how long will it take for Google to stop indexing that URL?
          GastonRiera
          GastonRiera
          0
          3
          98

        • Does anyone know how to appear with snippet that says something like: Jobs 1-10 of 80 in the beginning of the description on Google? e.g. like on: https://www.google.co.za/#q=pickers+and+packers
          Everett
          Everett
          0
          4
          151

        • Blocking out specific URLs with robots.txt
          Modi
          Modi
          0
          3
          133

        • Can URLs blocked with robots.txt hurt your site?
          workzentre
          workzentre
          0
          4
          302

        • Robots.txt: Can you put a /* wildcard in the middle of a URL?
          irvingw
          irvingw
          0
          2
          410

        • Search Engine Blocked by robots.txt for Dynamic URLs
          KeriMorgret
          KeriMorgret
          0
          2
          689

        • Blocking Dynamic URLs with Robots.txt
          TaitLarson
          TaitLarson
          1
          4
          5.1k

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy