The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Getting Started
    4. Moz site crawl doesn't work

    Moz site crawl doesn't work

    Getting Started
    4 3 82
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DrainKing
      DrainKing last edited by

      The Moz site crawl isn't working for my campaign, but works for the  site's on demand crawl. The search should not be disallowed by robots.txt or the headers. I'd like to be able to track the website for the campaign so I can see SEO gains / losses and increases / decreases in indexing.

      1 Reply Last reply Reply Quote 0
      • Xiano
        Xiano last edited by

        Does it give a warning/error, or is it simply returning no pages?

        1 Reply Last reply Reply Quote 0
        • samantha.chapman
          samantha.chapman last edited by

          Hi there,

          Sam from Moz's Help Team here - could you please pop a message about this over to help@moz.com along with the affected campaign name so we can look into this for you?

          Thank you!

          1 Reply Last reply Reply Quote 0
          • DrainKing
            DrainKing last edited by

            Hey guys,

            It does give an error:

            **Moz was unable to crawl your site on May 3, 2019. **Our crawler was banned by a page on your site, either through your robots.txt, the X-Robots-Tag HTTP header, or the meta robots tag. Update these tags to allow your page and the rest of your site to be crawled. If this error is found on any page on your site, it prevents our crawler (and some search engines) from crawling the rest of your site. Typically errors like this should be investigated and fixed by the site webmaster. Read our troubleshooting guide.

            With that in mind, it does not appear that the crawl should be blocked with the resources in place.

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • When I crawl my site On Moz it says it can't access the robots.txt file, but crawl is fine on SEM Rush - Anyone know any reason for this?
              meghanpahinui
              meghanpahinui
              0
              3
              60

            • Moz can't crawl my site.
              eli.myers
              eli.myers
              0
              2
              118

            • Why does the MOZ bar not work on SERPs with site links?
              samantha.chapman
              samantha.chapman
              0
              2
              73

            • I haven't used Moz and you've been billing me. Please refund thanks
              BlueCorona
              BlueCorona
              0
              3
              126

            • My question is, when you translate your website to another language, does moz crawl both or do i have to add another campaign to moz so that they can crawl it seperately?
              RyanPurkey
              RyanPurkey
              0
              4
              130

            • Why can't I Ctrl + click on links on Moz any more?
              max.favilli
              max.favilli
              1
              2
              96

            • Custom report doesn't list keywords.
              ukandyh
              ukandyh
              0
              2
              27

            • Why I can't add more campaign to my moz analytics?
              Futura
              Futura
              0
              8
              283

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy