The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Site Crawl Error

    Site Crawl Error

    Moz Tools
    5 3 434
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • nopsts
      nopsts last edited by

      In moz crawling error this message is appears:

      MOST COMMON ISSUES

      1Search Engine Blocked by robots.txt

      Error Code 612: Error response for robots.txt

      i asked help staff but they crawled again and nothing changed.

      there's only robots.XML (not TXT)  in root of my webpage it contains:

      User-agent: *
      Allow: /
      Allow: /sitemap.htm

      anyone please help me? thank you

      1 Reply Last reply Reply Quote 0
      • justin-brock
        justin-brock last edited by

        Would you mind sharing a link to your site?

        nopsts 1 Reply Last reply Reply Quote 0
        • nopsts
          nopsts @justin-brock last edited by

          This post is deleted!
          1 Reply Last reply Reply Quote 0
          • Abe_Schmidt
            Abe_Schmidt last edited by

            Hello, this is Abe from the Moz Help team. After our reported fix in your latest support ticket, you will see correct crawl crawl results in your next weekly update 🙂

            nopsts 1 Reply Last reply Reply Quote 0
            • nopsts
              nopsts @Abe_Schmidt last edited by

              Thank you,

              per your instructions i have changed robots.txt exactly like:

              User-agent: *
              Disallow:

              As you can see there's no Allow: / and Disallow line is blank

              is that correct?

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Site redesign makes Moz Site Crawl go haywire
                mrtemper
                mrtemper
                1
                2
                19

              • Does anyone know the linking of hashtags on Wix sites does it negatively or postively impact SEO. It is coming up as an error in site crawls 'Pages with 404 errors' Anyone got any experience please?
                Nozzle
                Nozzle
                0
                5
                82

              • Site Crawl 4xx Errors?
                mostcg
                mostcg
                0
                3
                189

              • Crawl diagnostics up to date after Magento ecommerce site crawl?
                Whebb
                Whebb
                0
                2
                218

              • How often does seomoz crawl the site? Can you force a crawl at a specific time ?
                stewbuch1872
                stewbuch1872
                0
                6
                188

              • Why would my site return an error when using Open Site Explorer to crawl it?
                KeriMorgret
                KeriMorgret
                0
                5
                183

              • Does Open Site Explorer purposefully not crawl some sites?
                KeriMorgret
                KeriMorgret
                0
                4
                281

              • I have corrected the Problems in Crawl Diagnostics. When would it refresh/ re-crawl my site ?
                VarunBansal
                VarunBansal
                0
                3
                475

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy