The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Blocking URL's with specific parameters from Googlebot

    Blocking URL's with specific parameters from Googlebot

    Technical SEO Issues
    11 2 2.9k
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • AlanMosley
      AlanMosley last edited by

      dont think you are going to do it in robots.txt, rather do a 301 from mode=vote to non mode vote.

      If you dont know how to put this into practise, tell me what your site is built with, if it is ASP.NET, i will show you how to impliment, if not someone else should be able to help.

      aethereal 1 Reply Last reply Reply Quote 0
      • aethereal
        aethereal @AlanMosley last edited by

        Thanks for the reply. The site was developed using PHP, mySQL and Javascript. I was hoping there was a way to do it without getting programmers involved...

        AlanMosley aethereal 8 Replies Last reply Reply Quote 0
        • AlanMosley
          AlanMosley @aethereal last edited by

          Try this in robots.txt, I did not think that Google allows wild cards but i just read that they do.

          Disallow: /*mode=vote*

          or

          Disallow: /*mode=vote

          or

          Disallow: /*mode

          Then try in Google WMT to read with googlebot to see if it works.

          The first in the list seems right to me, but I have seen others do it the other ways.

          1 Reply Last reply Reply Quote 1
          • aethereal
            aethereal @aethereal last edited by

            I added "Disallow: /mode=vote" to the robots.txt file and also manually entered it on Crawler Access page, then clicked "Test" and no errors were reported. The WMT page states that robots.txt was last downloaded 16 hours ago so I'll wait until it picks the file up again and then check for any errors. Hopefully that will do trick 🙂

            1 Reply Last reply Reply Quote 0
            • AlanMosley
              AlanMosley @aethereal last edited by

              Wrong place, go to diagnostics, then look for fetch as googlebot

              1 Reply Last reply Reply Quote 0
              • aethereal
                aethereal @aethereal last edited by

                Tried "Fetch as Googlebot" in Diagnostics and it came back as "Success" so I guess the robots.txt directive is not working. I'm assuming it should have reported a failure message when attempting to fetch a URL containing "?mode=vote".

                1 Reply Last reply Reply Quote 0
                • AlanMosley
                  AlanMosley @aethereal last edited by

                  Yes correct, did you try the other formats?

                  1 Reply Last reply Reply Quote 0
                  • aethereal
                    aethereal @aethereal last edited by

                    Tried them all now and all come back with "Success"... May be I'll post in the WMT Forum and see if anyone can shed light on this problem. Thanks for your help Alan, it's much appreciated.

                    1 Reply Last reply Reply Quote 0
                    • aethereal
                      aethereal @aethereal last edited by

                      This post is deleted!
                      1 Reply Last reply Reply Quote 0
                      • AlanMosley
                        AlanMosley @aethereal last edited by

                        Good to hear, I am glad you perservered

                        1 Reply Last reply Reply Quote 1
                        • 1 / 1
                        • First post
                          Last post
                        • If I'm using a compressed sitemap (sitemap.xml.gz) that's the URL that gets submitted to webmaster tools, correct?
                          ThompsonPaul
                          ThompsonPaul
                          0
                          6
                          1.8k

                        • Parked former company's url on top of my existing url and that URL is showing in SERPs for my top keywords
                          Joelabarre
                          Joelabarre
                          0
                          7
                          120

                        • We have designed a new site and are in dilemna whether or not to change the site's URL structure or maybe few odd looking urls. How exactly do we go about the URL thing in toto? thanks in advance, any suggestion(s) would be dearly welcome
                          ShaunPhilips
                          ShaunPhilips
                          1
                          7
                          109

                        • What's the best way to handle Overly Dynamic Url's?
                          GKLA
                          GKLA
                          0
                          2
                          309

                        • How to Block Urls with specific components from Googlebot
                          RyanKent
                          RyanKent
                          0
                          4
                          625

                        • Will a "blog=example "parameter at the end of my URLs affect google's crawling them?
                          Intridea
                          Intridea
                          0
                          3
                          428

                        • Are URL's with trailing slash seen as two different URLs
                          seoug_2005
                          seoug_2005
                          0
                          3
                          805

                        • Url's don't want to show up in google. Please help?
                          RyanKent
                          RyanKent
                          0
                          5
                          587

                        Get started with Moz Pro!

                        Unlock the power of advanced SEO tools and data-driven insights.

                        Start my free trial
                        Products
                        • Moz Pro
                        • Moz Local
                        • Moz API
                        • Moz Data
                        • STAT
                        • Product Updates
                        Moz Solutions
                        • SMB Solutions
                        • Agency Solutions
                        • Enterprise Solutions
                        • Digital Marketers
                        Free SEO Tools
                        • Domain Authority Checker
                        • Link Explorer
                        • Keyword Explorer
                        • Competitive Research
                        • Brand Authority Checker
                        • Local Citation Checker
                        • MozBar Extension
                        • MozCast
                        Resources
                        • Blog
                        • SEO Learning Center
                        • Help Hub
                        • Beginner's Guide to SEO
                        • How-to Guides
                        • Moz Academy
                        • API Docs
                        About Moz
                        • About
                        • Team
                        • Careers
                        • Contact
                        Why Moz
                        • Case Studies
                        • Testimonials
                        Get Involved
                        • Become an Affiliate
                        • MozCon
                        • Webinars
                        • Practical Marketer Series
                        • MozPod
                        Connect with us

                        Contact the Help team

                        Moz logo
                        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                        • Accessibility
                        • Terms of Use
                        • Privacy