The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Blocking Google from Crawling Parameters

    Blocking Google from Crawling Parameters

    Technical SEO Issues
    7 2 1.1k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DanaDV
      DanaDV last edited by

      Hi guys:

      What is the best way to keep Google from crawling certain urls with parameters? I used the setting in Webmaster Tools, but that doesn't seem to be helping at all. Can I use robots.txt or some other method? Thanks!

      Some examples are:

      <colgroup><col width="797"></colgroup>

      www.mayer-johnson.com/category/assistive-technology?manufacturer=179
      www.mayer-johnson.com/category/assistive-technology?manufacturer=226

      www.mayer-johnson.com/category/assistive-technology?manufacturer=227

      <colgroup><col width="797"></colgroup>

      www.mayer-johnson.com/category/english-language-learners?condition=212
      www.mayer-johnson.com/category/english-language-learners?condition=213

      www.mayer-johnson.com/category/english-language-learners?condition=214

      <colgroup><col width="797"></colgroup>
      | www.mayer-johnson.com/category/english-language-learners?roles=164 |
      | www.mayer-johnson.com/category/english-language-learners?roles=165 |
      | www.mayer-johnson.com/category/english-language-learners?roles=197 |

      |

      |

      1 Reply Last reply Reply Quote 0
      • wissamdandan
        wissamdandan last edited by

        Yes you can block it thew robots.txt and also by Adding a rel="canonical" link in the page code itself will accomplish the task.

        Disallow: /category/english-language-learners/*?

        DanaDV 2 Replies Last reply Reply Quote 0
        • DanaDV
          DanaDV @wissamdandan last edited by

          Thanks. I didn't want to use rel="canonical" because there are thousands of variations of these parameters, and it would be time consuming, to say the least.

          1 Reply Last reply Reply Quote 0
          • DanaDV
            DanaDV @wissamdandan last edited by

            So, for example, it would look like this?:

            Disallow:?condition=

            Disallow: ?cat=

            wissamdandan DanaDV 3 Replies Last reply Reply Quote 0
            • wissamdandan
              wissamdandan @DanaDV last edited by

              No,

              Disallow ?condition=

              Disallow ?cat=

              Disallow ?instructional_level=

              1 Reply Last reply Reply Quote 0
              • DanaDV
                DanaDV @DanaDV last edited by

                Thanks,  Wissam!

                1 Reply Last reply Reply Quote 1
                • wissamdandan
                  wissamdandan @DanaDV last edited by

                  anytime Dana 🙂

                  1 Reply Last reply Reply Quote 0
                  • 1 / 1
                  • First post
                    Last post
                  • Blocking Google from telemetry requests
                    Martijn_Scheijbeler
                    Martijn_Scheijbeler
                    0
                    2
                    190

                  • Google Crawling Issues! How Can I Get Google to Crawl My Website Regularly?
                    libero_net
                    libero_net
                    1
                    8
                    142

                  • Blocked URL parameters can still be crawled and indexed by google?
                    anthonydnelson
                    anthonydnelson
                    0
                    6
                    1.7k

                  • Will blocking the Wayback Machine (archive.org) have any impact on Google crawl and indexing/SEO?
                    EricHess
                    EricHess
                    0
                    3
                    2.2k

                  • CDN Being Crawled and Indexed by Google
                    irvingw
                    irvingw
                    0
                    2
                    1.0k

                  • Google Search Parameters
                    RyanKent
                    RyanKent
                    0
                    5
                    499

                  • Will a "blog=example "parameter at the end of my URLs affect google's crawling them?
                    Intridea
                    Intridea
                    0
                    3
                    428

                  • Best blocking solution for Google
                    KeriMorgret
                    KeriMorgret
                    0
                    5
                    1.0k

                  Get started with Moz Pro!

                  Unlock the power of advanced SEO tools and data-driven insights.

                  Start my free trial
                  Products
                  • Moz Pro
                  • Moz Local
                  • Moz API
                  • Moz Data
                  • STAT
                  • Product Updates
                  Moz Solutions
                  • SMB Solutions
                  • Agency Solutions
                  • Enterprise Solutions
                  • Digital Marketers
                  Free SEO Tools
                  • Domain Authority Checker
                  • Link Explorer
                  • Keyword Explorer
                  • Competitive Research
                  • Brand Authority Checker
                  • Local Citation Checker
                  • MozBar Extension
                  • MozCast
                  Resources
                  • Blog
                  • SEO Learning Center
                  • Help Hub
                  • Beginner's Guide to SEO
                  • How-to Guides
                  • Moz Academy
                  • API Docs
                  About Moz
                  • About
                  • Team
                  • Careers
                  • Contact
                  Why Moz
                  • Case Studies
                  • Testimonials
                  Get Involved
                  • Become an Affiliate
                  • MozCon
                  • Webinars
                  • Practical Marketer Series
                  • MozPod
                  Connect with us

                  Contact the Help team

                  Join our newsletter
                  Moz logo
                  © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                  • Accessibility
                  • Terms of Use
                  • Privacy