The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. API
    4. Crawler unable to access pages

    Crawler unable to access pages

    API
    5 3 66
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 2Cubedie
      2Cubedie last edited by

      Hi crawler is unable to access site and crawl properly. Mainly for the backlink checker, it's producing no results

      There is nothing in the robots.txt file blocking crawler access.

      Any help is much appreciated as it's driving me crazy!

      1 Reply Last reply Reply Quote 0
      • donsilvernail
        donsilvernail last edited by

        Is there are specific errors being reported along with the crawler message or see any numeric errors on the crawl page?

        Thanks,

        Don

        1 Reply Last reply Reply Quote 0
        • dave.kudera
          dave.kudera last edited by

          Hey, Dave from the Help Team here

          Are you receiving this error when running your site through Open Site Explorer?  Can you please provide the URL that you are checking?

          thanks!

          2Cubedie 1 Reply Last reply Reply Quote 0
          • 2Cubedie
            2Cubedie @dave.kudera last edited by

            Hi.

            Can I send the url privately if possible?

            Thanks

            dave.kudera 1 Reply Last reply Reply Quote 0
            • dave.kudera
              dave.kudera @2Cubedie last edited by

              Hey!

              Sure you can write into help@moz.com and we will get you sorted out!

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Does Moz DotBor crawler is the slowest than earlier?
                albertomr
                albertomr
                0
                3
                21

              • Sitemaps and Indexed Pages
                ThompsonPaul
                ThompsonPaul
                0
                4
                100

              • Sitemaps and Indexed Pages
                Slumberjac
                Slumberjac
                0
                5
                301

              • API Access Change
                DavidLee
                DavidLee
                0
                2
                75

              • API - Internal Links to page and related metrics
                DavidLee
                DavidLee
                1
                2
                375

              • On page reports
                Hardley111
                Hardley111
                0
                7
                105

              • On-Page Reports showing old urls
                DavidLee
                DavidLee
                0
                3
                111

              • Top Pages metrics in OSE
                John-E-Turner
                John-E-Turner
                0
                3
                178

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy