The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Robots.txt Disallow: / in Search Console

    Robots.txt Disallow: / in Search Console

    Technical SEO Issues
    2 2 124
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • RAN_SEO
      RAN_SEO last edited by

      Two days ago I found out through search console that my website's Robots.txt has changed to

      User-agent: *
      Disallow: /

      When I check the robots.txt in the website it looks fine - I see its blocked just in search console( in the robots.txt tester).

      when I try to do fetch as google to the homepage I see its blocked. Any ideas why would robots.txt block my website? it was fine until the weekend.

      • before that, in the last 3 months I saw I had blocked resources in the website and I brought back pages with fetch as google.

      Any ideas?

      1 Reply Last reply Reply Quote 0
      • GastonRiera
        GastonRiera last edited by

        Hello Ran,

        Just to clarify, in search console, when you go to Crawl-> Robots.txt Tester and in the middle right clicking en el see live robots.txt, it doesnt show the correct file?

        It could be that google isnt recrawling the new robots.txt

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Robots.txt to disallow /index.php/ path
          Mikkehl
          Mikkehl
          0
          9
          7.1k

        • Googlebot does not obey robots.txt disallow
          Cyrus-Shepard
          Cyrus-Shepard
          0
          12
          1.4k

        • Disallow: /search/ in robots but soft 404s are still showing in GWT and Google search?
          timhatton
          timhatton
          0
          7
          374

        • Allow or Disallow First in Robots.txt
          Net66SEO
          Net66SEO
          0
          12
          27.8k

        • Should search pages be disallowed in robots.txt?
          MichaelWeisbaum
          MichaelWeisbaum
          0
          3
          335

        • Should I set up a disallow in the robots.txt for catalog search results?
          AlanBleiweiss
          AlanBleiweiss
          0
          5
          1.7k

        • Robots.txt and robots meta
          TheEspresseo
          TheEspresseo
          0
          5
          1.1k

        • Search Engine blocked by robots.txt
          Goetzman
          Goetzman
          0
          7
          1.2k

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy