The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Pro
    4. Amount of Pages Crawled Dropped Significantly

    Amount of Pages Crawled Dropped Significantly

    Moz Pro
    5 3 439
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • jeffmace
      jeffmace last edited by

      I am just wondering if something changed with the SEOMoz  crawler.  I was always getting 10,000 or near 10,000 pages crawled.  After the last two  crawls I am ending up around 2500 pages.  Has anything changed that I would need to look at it see if I am blocking the crawler or something else?

      1 Reply Last reply Reply Quote 0
      • OptimizeSmart
        OptimizeSmart last edited by

        you may  be blocking the mozbot. Do you remember the exact date when it started happening? If yes then you can identify and revert the changes. Also check your robots.txt and .htaccess file for any abnormal changes during this time period.

        jeffmace 2 Replies Last reply Reply Quote 0
        • jeffmace
          jeffmace @OptimizeSmart last edited by

          i did a site update about 6 weeks ago, but was crawled a few times with about 9,000-10,000 pages crawled. It was the last two crawls that it dropped to 2504 pages.  I don't use .htaccess, nothing in robots.txt so I am not sure what could be doing it

          1 Reply Last reply Reply Quote 0
          • jeffmace
            jeffmace @OptimizeSmart last edited by

            i also want to add that I ran a test crawl and it actually returned 3200 pages, so I am not sure why the site is only doing 2504 for the last two weeks.

            1 Reply Last reply Reply Quote 0
            • KeriMorgret
              KeriMorgret last edited by

              Hi! This is probably a question for the help desk team. If you send them an email at help@seomoz.org and let them know your campaign name, they'll take a look at it and help you figure out what happened.

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Moz crawl only shows 2 pages, but we have more than 1000 pages.
                DavidLee
                DavidLee
                0
                8
                209

              • My moz only one page was crawled
                SmartVapes
                SmartVapes
                0
                5
                159

              • Crawl Diagnostics - Crawling way more pages than my site has?
                LodestoneGen
                LodestoneGen
                0
                5
                410

              • "Issue: Duplicate Page Content " in Crawl Diagnostics - but sample pages are not related to page indicated with duplicate content
                cbielich
                cbielich
                0
                2
                345

              • Why did SEOMoz only crawl 1 page?
                KeriMorgret
                KeriMorgret
                0
                5
                439

              • Drop in number of Pages crawled by Moz crawler
                Nick_Sayers
                Nick_Sayers
                0
                5
                412

              • Issue in number of pages crawled
                Marcus_Miller
                Marcus_Miller
                0
                4
                485

              • Only 1 page has been crawled. Why?
                tompollard
                tompollard
                0
                5
                557

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy