The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Search Engine Trends
    4. Google cant read my robots.txt from past 10 days

    Google cant read my robots.txt from past 10 days

    Search Engine Trends
    5 3 152
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • toxicpls
      toxicpls last edited by

      http://awesomescreenshot.com/08d1s6aybc

      hi, my robots.txt is

      http://wallpaperzoo.com/robots.txt

      google says it cant read and has postponed the crawl.. its been 10 days and no crawl.. please help me in solving this issue.. this is save with

      http://hdwallpaperzones.com/robots.txt

      1 Reply Last reply Reply Quote 0
      • StreamlineMetrics
        StreamlineMetrics last edited by

        There's nothing wrong with your robots.txt file as far as I can see. It is likely that Google tried accessing the robots.txt file when your server was slow/down for a split second and that's why you are seeing the error. Are you sure that the site hasn't been crawled in 10 days? Have you checked your server log files to verify that? Regardless, once Googlebot does eventually come back, it should read the robots.txt file correctly and then you should be good to go.

        toxicpls 1 Reply Last reply Reply Quote 1
        • toxicpls
          toxicpls @StreamlineMetrics last edited by

          hi sir, there was a problem on 23rd september in robots.txt which i fixed on 25th after then google bot dint come.. how much time it will take to come back again?

          Martijn_Scheijbeler 1 Reply Last reply Reply Quote 0
          • Martijn_Scheijbeler
            Martijn_Scheijbeler @toxicpls last edited by

            What you can do is from within Google Webmaster Tools is to ask Google to fetch the URL of the sitemap and then 'submit it to the index'. That's one technique I've seen working before.

            Google Webmaster Tools or Googlebot doesn't seem to crawl robots.txt files very often. Have seen it a couple of times where it could take days for them to crawl the file again after I've changed something.

            1 Reply Last reply Reply Quote 1
            • toxicpls
              toxicpls last edited by

              Hi sir Google started fetching again

              🙂 thank you for your help

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Adding the link masking directory to robots.txt?
                Everett
                Everett
                0
                3
                243

              • Google Rankings Dropped in Past Few Weeks
                William.Lau
                William.Lau
                0
                3
                154

              • Has anyone experienced a dramatic decrease in Google rankings followed by a dramatic increase in the past few days?
                BlueprintMarketing
                BlueprintMarketing
                0
                4
                175

              • Someone just told me that the Google doesn't read past the pipe symbol. I find that hard to believe. Is this true?
                MarketingAgencyFlorida
                MarketingAgencyFlorida
                0
                6
                361

              • Why would Google read different pages to rank for a keyword?
                Solvari
                Solvari
                0
                2
                257

              • More youtube results from the past 2 days
                Nobody1560986989723
                Nobody1560986989723
                0
                2
                289

              • Lost 50% google traffic in one day - panic?
                csfarnsworth
                csfarnsworth
                1
                81
                7.1k

              • Did google change their algorithm over the past week?
                Websensejim
                Websensejim
                0
                3
                784

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy