The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Is it possible to block Moz from crawling sites?

    Is it possible to block Moz from crawling sites?

    Moz Tools
    4 3 239
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • co.mc
      co.mc last edited by

      Hi, is it possible to stop Moz from crawling a site at the server level? Not that I am looking to do this or anything, but here's why I'm asking.

      I have been crawling a site that is managed (currently by 2 parties), and I noticed that this week pages crawled went from 80 (last week) to 1 page!! I know, what? See my image attached... and the issues all went to zero "0"....!

      So is it possible that someone can't prevent Moz from crawling the site at the server level? I checked the robots.txt file on the site, but nothing there. I'm curious.

      dYNUwjd.jpg

      1 Reply Last reply Reply Quote 0
      • MoosaHemani
        MoosaHemani last edited by

        Ok, here is the helpful resource regarding Roger Mozbot. http://moz.com/help/pro/rogerbot-crawler

        This might help!

        1 Reply Last reply Reply Quote 1
        • KeriMorgret
          KeriMorgret last edited by

          Here are the most common reasons why only one page is crawled. If this doesn't answer it, send us a note at help@moz.com. Thanks!

          https://seomoz.zendesk.com/entries/409821-Why-Isn-t-My-Site-Being-Crawled-You-re-Not-Crawling-All-My-Pages-

          1 Reply Last reply Reply Quote 1
          • co.mc
            co.mc last edited by

            Thank you both for the responses... I guess it was just a matter of time in my case. It looks like the keywords crawl/report comes in a few hours "after" the initial crawl email from Moz... not sure why, but about 15 minutes after I posted this question, I received the email titled "New Rankings and On-Page Reports Ready" that included the keyword rankings!

            Thanks again

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Hello There I have a site the problem with this site is I have taken countless backlinks but it is not reflecting on Moz. Is there any way to help Moz find my links?
              BartonInteractive
              BartonInteractive
              0
              4
              59

            • Block Moz (or any other robot) from crawling pages with specific URLs
              Blacktie
              Blacktie
              0
              3
              844

            • Possible Crawling Problem with Screaming Frog and Moz Crawlers
              CJ5
              CJ5
              0
              4
              702

            • I need an interlinking report for my site, is there a report in Moz or another application that tell me how all of my pages are linked to other pages on my site?
              Linda-Vassily
              Linda-Vassily
              0
              3
              132

            • Seo moz has only crawled 2 pages of my site. Ive been notified of a 403 error and need an answer as to why my pages are not being crawled?
              nitro-digital
              nitro-digital
              0
              9
              319

            • How often does seomoz crawl the site? Can you force a crawl at a specific time ?
              stewbuch1872
              stewbuch1872
              0
              6
              188

            • Why would my site return an error when using Open Site Explorer to crawl it?
              KeriMorgret
              KeriMorgret
              0
              5
              183

            • SEO Moz analysing US site when I want it to analyse UK site
              Peterli
              Peterli
              1
              5
              223

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy