The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Search Results Pages Blocked in Robots.txt?

    Search Results Pages Blocked in Robots.txt?

    Intermediate & Advanced SEO
    3 2 117
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • BeckyKey
      BeckyKey last edited by

      Hi

      I am reviewing our robots.txt file. I wondered if search results pages should be blocked from crawling?

      We currently have this in the file

      /searchterm*
      
      Is it a good thing for SEO?
      
      1 Reply Last reply Reply Quote 0
      • DirkC
        DirkC last edited by

        It's probably a good thing - I would keep them blocked.

        Check https://www.mattcutts.com/blog/search-results-in-search-results/ - quote "Use robots.txt to prevent crawling of search results pages or other auto-generated pages that don’t add much value for users coming from search engines."

        Dirk

        BeckyKey 1 Reply Last reply Reply Quote 2
        • BeckyKey
          BeckyKey @DirkC last edited by

          HI Dirk,

          Sorry I missed this one, thanks for your input 🙂

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Scary bug in search console: All our pages reported as being blocked by robots.txt after https migration
            Christy-Correll
            Christy-Correll
            0
            6
            94

          • If Robots.txt have blocked an Image (Image URL) but the other page which can be indexed has this image, how is the image treated?
            alphonseha
            alphonseha
            1
            4
            1.4k

          • Pages getting into Google Index, blocked by Robots.txt??
            Devanur-Rafi
            Devanur-Rafi
            0
            10
            673

          • "No index" page still shows in search results and paginated pages shows page 2 in results
            khi5
            khi5
            0
            3
            114

          • Does using robots.txt to block pages decrease search traffic?
            KeriMorgret
            KeriMorgret
            0
            4
            520

          • Search Engine Blocked by robots.txt for Dynamic URLs
            KeriMorgret
            KeriMorgret
            0
            2
            689

          • Old pages still crawled by SE returning 404s. Better to put 301 or block with robots.txt ?
            RyanKent
            RyanKent
            0
            2
            721

          • Category Pages - Canonical, Robots.txt, Changing Page Attributes
            Function5
            Function5
            0
            9
            1.1k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy