The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Does Google crawl the pages which are generated via the site's search box queries?

    Does Google crawl the pages which are generated via the site's search box queries?

    Intermediate & Advanced SEO
    5 4 606
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • pulseseo
      pulseseo last edited by

      For example, if I search for an 'x' item in a site's search box and if the site displays a list of results based on the query, would that page be crawled? I am asking this question because this would be a URL that is non existent on the site and hence am confused as to whether Google bots would be able to find it.

      1 Reply Last reply Reply Quote 0
      • SuperlativB
        SuperlativB last edited by

        "if I search for an 'x' item in a site's search box and if the site displays a list of results based on the query, would that page be crawled?"
        Is it google search-box for sites your talking about?

        " I am asking this question because this would be a URL that is non existent on the site"
        **If it does'nt exist you wouldent find it? or do you mean that the page is not to find in the menu-navigation? **

        pulseseo 1 Reply Last reply Reply Quote 0
        • pulseseo
          pulseseo @SuperlativB last edited by

          No I am not talking about the google search box incorporated in sites but the site's own search box. Answering your 2nd question, I meant that the URL won't be found via site navigation as it is a dynamic URL that is generated. Awaiting your response on the same.

          1 Reply Last reply Reply Quote 0
          • zeepartner
            zeepartner last edited by

            Google could crawl the dynamic URLs created by your searchbox - but it usually doesn't unless there is a link to such a dynamic url somewhere. Internal searches don't create much problems anymore, but if you want to be sure, you could always block your dynamic search results pages via robots.txt or Google Webmaster Tools (>Site configuration >URL parameters).

            So if the URL generated by internal searches is http://www.site.com/search/?searchword=search+query+here, you could add this to robots.txt:

            User-agent: *

            Disallow: /search/

            1 Reply Last reply Reply Quote 1
            • YannickVeys
              YannickVeys last edited by

              Google crawls the pages. Google sometimes even tries the search box by typing in a random word to see what happens.

              Have a look at this url: https://www.google.com/search?q=site%3Agoogle.com%20inurl%3A%22search%3Fq%22 You'll see that search queries have been indexed. It happens if people link to it. It doesn't matter that the url is "non existent". It actually is, because it doesn't display a 404 error or any other error. It gives a 200 OK code, so search engines see it as a normal page. Google will probably not index a page it "makes" by filling in a random search term, but will index such a page when it is linked to.

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • How long will old pages stay in Google's cache index. We have a new site that is two months old but we are seeing old pages even though we used 301 redirects.
                DonnaDuncan
                DonnaDuncan
                0
                3
                81

              • What can cause for a service page to rank in Google's Answer Box?
                eukmark
                eukmark
                0
                5
                148

              • Some site's links look different on google search. For example Games.com › Flash games › Decoration games How can we do our url's like this?
                davebuts
                davebuts
                0
                4
                268

              • Investigating Google's treatment of different pages on our site - canonicals, addresses, and more.
                MiriamEllis
                MiriamEllis
                0
                2
                56

              • Big discrepancies between pages in Google's index and pages in sitemap
                David-Kley
                David-Kley
                0
                6
                218

              • How to remove my site's pages in search results?
                CleverPhD
                CleverPhD
                0
                7
                108

              • I want to block search bots in crawling all my website's pages expect for homepage. Is this rule correct?
                GPainter
                GPainter
                0
                7
                135

              • What's the best way to check Google search results for all pages NOT linking to a domain?
                Cyrus-Shepard
                Cyrus-Shepard
                0
                7
                163

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy