The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. SEOmoz crawler not crawling my site

    SEOmoz crawler not crawling my site

    Moz Tools
    5 2 787
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • IMM
      IMM last edited by

      We set up a new campaign in SEOmoz on Friday. It is my understanding that the preliminary crawl can cover up to 250 and this has been our experience in the past. However, the preliminary crawl only went through 2 pages. This is a larger eCommerce site with many pages. Any ideas why more pages weren't crawled?

      We set up the campaign to track at the root domain level.

      1 Reply Last reply Reply Quote 0
      • RuthBurrReedy
        RuthBurrReedy last edited by

        Hmm! Well, it could be a couple of things.

        If your full crawl ends up crawling the whole site, it's possible that the preliminary crawl timed out on a request to your site. As long as you continue seeing the full crawl, you should be good to go.

        If your next crawl continues showing just a few pages, I would check to make sure you're not blocking robots with either the meta robots tag, a robots.txt file, or nofollowed links. If you don't uncover anything out of the ordinary there, it could be an account- or campaign-specific error on our end; please email help@seomoz.org and our super-awesome Help Team can help get that fixed.

        1 Reply Last reply Reply Quote 1
        • IMM
          IMM last edited by

          Thanks for the reply Ruth!

          I was a little unclear as to part of your response. What were you referring to when you said "as long as you continue seeing the full crawl, you should be good to go"?

          Where should I be seeing the full crawl? When I look at the Crawl Diagnostic Summary, it notes that the last crawl was on May 10, 2013 (the day we set up the campaign) and the next crawl is to start May 18, 2013. I received email notification that SEOmoz was successfully recrawled it May 11, 2013. Similar to the May 10 crawl, the May 11 crawl only went through 2 pages.

          I also checked up on our robots.txt and tags. There isn't blocking anything the crawler on that front.

          RuthBurrReedy 1 Reply Last reply Reply Quote 0
          • RuthBurrReedy
            RuthBurrReedy @IMM last edited by

            So that's really what I meant was "as long as you're seeing the full crawl," was that if your next crawl found the whole site it would probably be fine. It sounds like that's not the case, so I think the Help Team is your best bet - they're far more able to diagnose campaign-specific issues than I am 🙂

            1 Reply Last reply Reply Quote 1
            • IMM
              IMM last edited by

              Thanks, Ruth. I see what you're saying about the crawl now. I'll go ahead and reach out the help team. Hopefully we can figure it out!

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • How do a run a MOZ crawl of my site before waiting for the scheduled weekly crawl?
                Kingalan1
                Kingalan1
                0
                6
                654

              • Does Open Site Explorer purposefully not crawl some sites?
                KeriMorgret
                KeriMorgret
                0
                4
                281

              • Why does SEOMoz only crawl 1 page of my site?
                KeriMorgret
                KeriMorgret
                0
                2
                329

              • How long would a SEOMoz crawl usually take for a site with around 4000 pages?
                Cyrus-Shepard
                Cyrus-Shepard
                0
                3
                653

              • Crawl test tool from SEOmoz - which URLs does it actually crawl?
                ShaMenz
                ShaMenz
                0
                4
                787

              • How to get seomoz to re-crawl a site?
                KeriMorgret
                KeriMorgret
                0
                5
                1.1k

              • I put a crawl on my site via seomoz and has come back saying only 1 page has been crawled.It has been over a week now can anyone help?
                jaytwotwenty
                jaytwotwenty
                0
                3
                739

              • SEOMoz site crawlers created an issue for our servers
                G-Force
                G-Force
                0
                4
                992

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy