The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. API
    4. Crawler unable to access pages

    Crawler unable to access pages

    API
    5 3 66
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • 2Cubedie
      2Cubedie last edited by

      Hi crawler is unable to access site and crawl properly. Mainly for the backlink checker, it's producing no results

      There is nothing in the robots.txt file blocking crawler access.

      Any help is much appreciated as it's driving me crazy!

      1 Reply Last reply Reply Quote 0
      • donsilvernail
        donsilvernail last edited by

        Is there are specific errors being reported along with the crawler message or see any numeric errors on the crawl page?

        Thanks,

        Don

        1 Reply Last reply Reply Quote 0
        • dave.kudera
          dave.kudera last edited by

          Hey, Dave from the Help Team here

          Are you receiving this error when running your site through Open Site Explorer?  Can you please provide the URL that you are checking?

          thanks!

          2Cubedie 1 Reply Last reply Reply Quote 0
          • 2Cubedie
            2Cubedie @dave.kudera last edited by

            Hi.

            Can I send the url privately if possible?

            Thanks

            dave.kudera 1 Reply Last reply Reply Quote 0
            • dave.kudera
              dave.kudera @2Cubedie last edited by

              Hey!

              Sure you can write into help@moz.com and we will get you sorted out!

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Does Moz DotBor crawler is the slowest than earlier?
                albertomr
                albertomr
                0
                3
                21

              • Invalid Access ID or Secret Key
                effectdigital
                effectdigital
                0
                3
                151

              • Navigation pages with a PA 1
                eli.myers
                eli.myers
                0
                4
                54

              • API ACCESS to COMPETITOR DATA
                Martijn_Scheijbeler
                Martijn_Scheijbeler
                0
                3
                218

              • Sitemaps and Indexed Pages
                Slumberjac
                Slumberjac
                0
                5
                301

              • Paid access to the API
                LisaHunt
                LisaHunt
                0
                2
                165

              • Does Moz's crawlers use _escaped_fragment_ to inspect pages on a single-page application?
                jameskais
                jameskais
                0
                5
                365

              • Huge in crease in on page errors
                MattRoney
                MattRoney
                0
                17
                198

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy