The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Robots.txt - "File does not appear to be valid"

    Robots.txt - "File does not appear to be valid"

    Technical SEO Issues
    3 2 319
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • PeaSoupDigital
      PeaSoupDigital last edited by

      Good afternoon Mozzers!

      I've got a weird problem with one of the sites I'm dealing with. For some reason, one of the developers changed the robots.txt file to disavow every site on the page - not a wise move!

      To rectify this, we uploaded the new robots.txt file to the domain's root as per Webmaster Tool's instructions. The live file is: User-agent: * (http://www.savistobathrooms.co.uk/robots.txt)

      I've submitted the new file in Webmaster Tools and it's pulling it through correctly in the editor. However, Webmaster Tools is not happy with it, for some reason. I've attached an image of the error.

      Does anyone have any ideas? I'm managing another site with the exact same robots.txt file and there are no issues.

      Cheers,

      Lewis

      FNcK2YQ

      1 Reply Last reply Reply Quote 0
      • PatrickDelehanty
        PatrickDelehanty last edited by

        Hi there

        I want to say that needs an...

        Allow: /

        ...or a "Group 2" specification.

        I would take a look at Google Developer's Robots.txt Specifications and see where you have opportunities to remedy this issue.

        Hope this helps! Good luck!

        1 Reply Last reply Reply Quote 3
        • PeaSoupDigital
          PeaSoupDigital last edited by

          Thanks for the quick response, Patrick. Why, if this robots.txt file is incorrect, does it yield no errors on other sites we use this on?

          Cheers,

          Lewis

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Blocking subdomains with Robots.txt file
            PaulM01
            PaulM01
            0
            3
            641

          • Google Webmaster Tools is saying "Sitemap contains urls which are blocked by robots.txt" after Https move...
            vetofunk
            vetofunk
            0
            5
            11.2k

          • "Url blocked by robots.txt." on my Video Sitemap
            sergeystefoglo
            sergeystefoglo
            0
            3
            521

          • "Extremely high number of URLs" warning for robots.txt blocked pages
            KristinaKledzik
            KristinaKledzik
            0
            8
            380

          • Does this robots.txt file look right?
            ThompsonPaul
            ThompsonPaul
            0
            8
            188

          • Do i have my robots.txt file set up properly
            ClaireH-184886
            ClaireH-184886
            1
            4
            319

          • Same URL in "Duplicate Content" and "Blocked by robots.txt"?
            alsvik
            alsvik
            0
            3
            502

          • Robots.txt File Redirects to Home Page
            NickPateman81
            NickPateman81
            0
            6
            4.1k

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy