The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. API
    4. Crawler unable to access pages

    Crawler unable to access pages

    API
    5 3 66
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 2Cubedie
      2Cubedie last edited by

      Hi crawler is unable to access site and crawl properly. Mainly for the backlink checker, it's producing no results

      There is nothing in the robots.txt file blocking crawler access.

      Any help is much appreciated as it's driving me crazy!

      1 Reply Last reply Reply Quote 0
      • donsilvernail
        donsilvernail last edited by

        Is there are specific errors being reported along with the crawler message or see any numeric errors on the crawl page?

        Thanks,

        Don

        1 Reply Last reply Reply Quote 0
        • dave.kudera
          dave.kudera last edited by

          Hey, Dave from the Help Team here

          Are you receiving this error when running your site through Open Site Explorer?  Can you please provide the URL that you are checking?

          thanks!

          2Cubedie 1 Reply Last reply Reply Quote 0
          • 2Cubedie
            2Cubedie @dave.kudera last edited by

            Hi.

            Can I send the url privately if possible?

            Thanks

            dave.kudera 1 Reply Last reply Reply Quote 0
            • dave.kudera
              dave.kudera @2Cubedie last edited by

              Hey!

              Sure you can write into help@moz.com and we will get you sorted out!

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Page Authority 2.0 is coming soon!
                eli.myers
                eli.myers
                8
                29
                926

              • API ACCESS to COMPETITOR DATA
                Martijn_Scheijbeler
                Martijn_Scheijbeler
                0
                3
                218

              • Number of Pages Crawled dropped significantly
                tawnycase
                tawnycase
                0
                2
                72

              • Sitemaps and Indexed Pages
                ThompsonPaul
                ThompsonPaul
                0
                4
                100

              • Sitemaps and Indexed Pages
                Slumberjac
                Slumberjac
                0
                5
                301

              • Bulk Page Authority Tracking
                BeckyKey
                BeckyKey
                0
                11
                1.1k

              • On-Page Reports showing old urls
                DavidLee
                DavidLee
                0
                3
                111

              • Huge in crease in on page errors
                MattRoney
                MattRoney
                0
                17
                198

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy