The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Link Explorer
    4. Moz crawling doesn't show all of my pages

    Moz crawling doesn't show all of my pages

    Link Explorer
    4 3 226
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • MK-Discoolver
      MK-Discoolver last edited by

      Hello,

      I'm trying to make an SEO on-pages audit of my website.

      When using the SEO crawl, i see only one page while i have much more pages in this website (200+). Is it an issue with the sitemap ? The robot.txt ?

      How can i check and correct this. My website is discoolver.com

      1 Reply Last reply Reply Quote 1
      • HashtagHustler
        HashtagHustler last edited by

        So just looking it appears you don't have a robots.txt file or a sitemap. At least one that I could easily find.

        This Moz post might help when creating one.

        1 Reply Last reply Reply Quote 0
        • MK-Discoolver
          MK-Discoolver last edited by

          Thanks for the answers.

          I believe the issue is about a rel=canonical from  the homepage which links to an external website...

          Can this tag prevent a good website crawl ?

          1 Reply Last reply Reply Quote 0
          • samantha.chapman
            samantha.chapman last edited by

            Hi there,

            Sam from Moz's Help Team here!

            So, I took a look at your site, and unfortunately I believe this is a result of our crawler's inability to work with Javascript.  Any client side loaded content will not be rendered to our crawler as we are only seeing what is rendered by your server.

            Our crawler works by parsing the source code of your site, looking at HTML elements.  If your site is primarily Javascript, then the data you get back with regards to the crawl report won't be completely accurate because of this. There's no real workaround that I can recommend for this one, since it is a technical limitation of our tools, but there are some good blog posts and discussions in the Q&A about this if you head over to our Help Hub.

            While the tools and data that rely on our crawl of your site may not return the best results because of that Javascript, your keyword rankings and link profile should work just fine.

            You might also want to check out a few tools that are compatible with Javascript, like Botify or Screaming Frog.

            I'm really sorry I can't be of more help here; Please let me know if there's anything else I can help with, or if there are any follow-up questions you might have!

            1 Reply Last reply Reply Quote 0
            • 1 / 1
            • First post
              Last post
            • Moz can't crawl our site
              Natalie-Alexis
              Natalie-Alexis
              0
              2
              22

            • Moz isn't crawling all my backlinks.
              KristinaKeyser
              KristinaKeyser
              0
              8
              73

            • Why doesn't Moz crawler follow robots.txt?
              Tylerj
              Tylerj
              0
              10
              879

            • Why isn't OSE showing any of my links?
              DanDeceuster
              DanDeceuster
              0
              5
              272

            • Moz can't crawl domain due to IP Geo redirect loop
              TomVolpe
              TomVolpe
              0
              2
              411

            • Why doesn't my site show up in Moz Site Explorer
              SamWeber
              SamWeber
              0
              3
              311

            • Why Moz doesn't offer tool similar like SEOprofiler Link Disinfection?
              Futura
              Futura
              0
              4
              1.8k

            • How Is a Page Crawled by Moz When Moz Says 'No Links'?
              xerox432
              xerox432
              0
              9
              153

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy