The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. On-Page / Site Optimization
    4. Googlebot found an extremely high number of URLs on your site:

    Googlebot found an extremely high number of URLs on your site:

    On-Page / Site Optimization
    5 3 222
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Obbserv
      Obbserv last edited by

      Website: www.gobol.in

      Although I have no indexed my search pages by adding /catalogsearch in robots.txt, still we are getting same error again and again

      Here's a list of sample URLs with potential problems.

      • http://www.gobol.in/catalogsearch/result/index/?category=&mobile_feature=4575_4578&q=panasonic+NR-BU303LH1H+REFRIGERATOR+296+L+GREY&special_price=32%2C456&x=0&y=0
      • http://www.gobol.in/mobile-and-accessories/mobiles-and-brands.html?manufacturer=4753_3355_455_4435_4720_3407_2412_4728_4784_4790_2010_4789_4376_2469&operating_system_mobile=4612

      Please help

      1 Reply Last reply Reply Quote 0
      • Martijn_Scheijbeler
        Martijn_Scheijbeler last edited by

        Hi Ravi,

        This message you got from Google is more a warning then an error, that is good to keep in mind. As there is no problem in theory with having a lot of URLs on your site if the quality is high enough. Therefore the first URL shouldn't cause any troubles, as like you mention, it's included in the robots.txt file.

        The second URL is a bit less nice but shouldn't also be trouble, as it's making sure that the URL parameters are working. Also there is a canonical URL on all these filter pages to make sure the pages are originally linked back to the category page. It could be worth it to check the URL parameters within Google Webmaster Tools to make sure Google is aware of all the changes in content for different parameters.

        Obbserv 1 Reply Last reply Reply Quote 1
        • Obbserv
          Obbserv @Martijn_Scheijbeler last edited by

          Thanks Martijn

          1 Reply Last reply Reply Quote 0
          • DirkC
            DirkC last edited by

            Hi Ravi,

            Did you use the robots.txt tester in Webmastertools - your robots.txt mentions to block /catalogsearch - if you want to block the entire directory I think you have to use

            Disallow: /catalogsearch/

            On top of that /catalogsearch is only blocked for Googlebot-image - not for the "normal" Googlebot - if you want to have it blocked for all bots you have to move it to the beginning of the file. Idem for /admin => should be /admin/ and in the beginning of your robots.txt file

            rgds,

            Dirk

            Obbserv 1 Reply Last reply Reply Quote 2
            • Obbserv
              Obbserv @DirkC last edited by

              Thanks Dirk

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Reducing number of site pages?
                CalicoKitty2000
                CalicoKitty2000
                0
                5
                153

              • To avoid the duplicate content issue I have created new urls for that specific site I am posting to and redirecting that url to the original on my site. Is this the right way to do it?
                0
                1
                24

              • Multi Keyword URL Ranking at Number 1
                MattRoney
                MattRoney
                0
                4
                101

              • Redirects for new site new urls?
                scott315
                scott315
                0
                6
                93

              • Tips on URL structure for a site re-design
                RyanCrawf1984
                RyanCrawf1984
                0
                7
                160

              • URL Structure for Home Decorating site
                SEO5Team
                SEO5Team
                0
                2
                244

              • Numbers in URL's - Search friendly or not?
                KeriMorgret
                KeriMorgret
                0
                3
                592

              • Site URL's
                portalseo
                portalseo
                0
                8
                1.4k

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy