The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Robots.txt question

    Robots.txt question

    Intermediate & Advanced SEO
    2 2 107
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • rbai
      rbai last edited by

      I notice something weird in Google robots. txt tester

      I have this line

      Disallow: display=

      in my robots.text but whatever URL I give to test it says blocked and shows this line in robots.text

      for example this line is to block pages like

      http://www.abc.com/lamps/floorlamps?display=table

      but if I test

      http://www.abc.com/lamps/floorlamps or any page

      it shows as blocked due to Disallow: display=

      am I doing something wrong or Google is just acting strange? I don't think pages with no display= are blocked in real.

      1 Reply Last reply Reply Quote 0
      • Mobilio
        Mobilio last edited by

        Yes - there is bug in your robots.txt. You should wrote some as:
        Disallow: /?display=table
        or:
        Disallow: /
        ?display=*

        1 Reply Last reply Reply Quote 5
        • 1 / 1
        • First post
          Last post
        • Twitter Robots.TXT
          MarketingChimp10
          MarketingChimp10
          0
          5
          453

        • Have a Robots.txt Issue
          MattRoney
          MattRoney
          0
          5
          226

        • Meta robots or robot.txt file?
          Andy.Drinkwater
          Andy.Drinkwater
          0
          5
          152

        • Robots.txt
          Travis_Bailey
          Travis_Bailey
          0
          4
          107

        • Robots.txt
          TomRayner
          TomRayner
          0
          5
          137

        • Robot.txt error
          Rubix
          Rubix
          0
          13
          203

        • Robots.txt Question
          Dr-Pete
          Dr-Pete
          0
          4
          231

        • Robots.txt unblock
          Elchanan
          Elchanan
          0
          5
          4.3k

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy