The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. I have 3500 pages crawled by Google, - why is SEOMOZ only able to crawl 400 of these ?

    I have 3500 pages crawled by Google, - why is SEOMOZ only able to crawl 400 of these ?

    Technical SEO Issues
    2 2 95
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • haybob27
      haybob27 last edited by

      I added my site almost two weeks ago to the PRO DashBoard, and so far only 404 pages has been crawled, - but I know for a fact that there is 3500 pages that should be crawled. Other search engines has no problem in crawling and indexing these pages, so what can be wrong here ?

      1 Reply Last reply Reply Quote 0
      • Abe_Schmidt
        Abe_Schmidt last edited by

        Hello Soren and thanks for writing us! It appears that your robots.txt is blocking quite a few folders and I am guessing that this could be part of the reason. If you can email us at help@seomoz.org with a couple examples of pages you are missing from your crawls, we would love to create a support ticket and dig in a bit further.

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Can view pages of site, but Google & SEOmoz return 404
          KeriMorgret
          KeriMorgret
          0
          6
          117

        • How do I keep Google from crawling my PPC landing page?
          chris.kent
          chris.kent
          0
          3
          422

        • Nofollow links appear to be still included in SEOMOZ crawl and Google
          adventure34
          adventure34
          0
          3
          354

        • Can SEOMoz crawl a single page as oppose to an entire subfolder?
          CollinJarman
          CollinJarman
          0
          6
          318

        • Google Webmaster tools vs SeoMOZ Crawl Diagnostics
          Yorkie
          Yorkie
          0
          5
          1.4k

        • SEOMoz Crawl Diagnostic indicates duplicate page content for home page?
          Linesides
          Linesides
          0
          3
          607

        • I have 15,000 pages. How do I have the Google bot crawl all the pages?
          KeriMorgret
          KeriMorgret
          0
          5
          714

        • SeoMoz robot is not able to crawl my website.
          G-Force
          G-Force
          0
          6
          730

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy