The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Is it possible to block Moz from crawling sites?

    Is it possible to block Moz from crawling sites?

    Moz Tools
    4 3 239
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • co.mc
      co.mc last edited by

      Hi, is it possible to stop Moz from crawling a site at the server level? Not that I am looking to do this or anything, but here's why I'm asking.

      I have been crawling a site that is managed (currently by 2 parties), and I noticed that this week pages crawled went from 80 (last week) to 1 page!! I know, what? See my image attached... and the issues all went to zero "0"....!

      So is it possible that someone can't prevent Moz from crawling the site at the server level? I checked the robots.txt file on the site, but nothing there. I'm curious.

      dYNUwjd.jpg

      1 Reply Last reply Reply Quote 0
      • MoosaHemani
        MoosaHemani last edited by

        Ok, here is the helpful resource regarding Roger Mozbot. http://moz.com/help/pro/rogerbot-crawler

        This might help!

        1 Reply Last reply Reply Quote 1
        • KeriMorgret
          KeriMorgret last edited by

          Here are the most common reasons why only one page is crawled. If this doesn't answer it, send us a note at help@moz.com. Thanks!

          https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages-

          1 Reply Last reply Reply Quote 1
          • co.mc
            co.mc last edited by

            Thank you both for the responses... I guess it was just a matter of time in my case. It looks like the keywords crawl/report comes in a few hours "after" the initial crawl email from Moz... not sure why, but about 15 minutes after I posted this question, I received the email titled "New Rankings and On-Page Reports Ready" that included the keyword rankings!

            Thanks again

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Site redesign makes Moz Site Crawl go haywire
              mrtemper
              mrtemper
              1
              2
              19

            • Block Moz (or any other robot) from crawling pages with specific URLs
              Blacktie
              Blacktie
              0
              3
              844

            • Possible Crawling Problem with Screaming Frog and Moz Crawlers
              CJ5
              CJ5
              0
              4
              702

            • How do a run a MOZ crawl of my site before waiting for the scheduled weekly crawl?
              Kingalan1
              Kingalan1
              0
              6
              654

            • Where is Moz Crawl?
              Christy-Correll
              Christy-Correll
              0
              4
              154

            • Why would my site return an error when using Open Site Explorer to crawl it?
              KeriMorgret
              KeriMorgret
              0
              5
              183

            • I have corrected the Problems in Crawl Diagnostics. When would it refresh/ re-crawl my site ?
              VarunBansal
              VarunBansal
              0
              3
              475

            • I put a crawl on my site via seomoz and has come back saying only 1 page has been crawled.It has been over a week now can anyone help?
              jaytwotwenty
              jaytwotwenty
              0
              3
              739

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy