The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Other Research Tools
    4. My page cant be crawled by MOZ

    My page cant be crawled by MOZ

    Other Research Tools
    8 5 109
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • zita.vn
      zita.vn last edited by

      Hi everyone,

      I have a page named zita.vn. Im using Moz to do SEO for the page, but when I use On-page grader in MOZ, I got an error that the page cant be accessed. Additionally, I got an issue with page crawling.

      Google search still crawl my page normally.

      Please help.

      Thanks a lot.

      1 Reply Last reply Reply Quote 0
      • Igor.Go
        Igor.Go last edited by

        Maybe you've set a number of threads too high and pause between requests too low? Some hosts will block IPs that send too many requests. How much is too many depends on your host.

        1 Reply Last reply Reply Quote 0
        • Mustansar
          Mustansar last edited by

          there is a possiblity of this issue is due to TLD, as i believe Moz crawler do not support to all TLDs as of yet. this could be the only reason comes to my mind. However, i reckon to raise a suppert ticket for more certian answer for this.  🙂

          1 Reply Last reply Reply Quote 0
          • vcj
            vcj last edited by

            I'm seeing a 403 response for your website:

            wget zita.vn
            --2016-08-02 11:46:09-- http://zita.vn/
            Resolving zita.vn... 107.167.188.173
            Connecting to zita.vn|107.167.188.173|:80... connected.
            HTTP request sent, awaiting response... 301 Moved Permanently
            Location: https://www.zita.vn/ [following]
            --2016-08-02 11:46:09-- https://www.zita.vn/
            Resolving www.zita.vn... 107.167.188.173
            Connecting to www.zita.vn|107.167.188.173|:443... connected.
            HTTP request sent, awaiting response... 403 Forbidden
            2016-08-02 11:46:10 ERROR 403: Forbidden.

            If Moz is seeing a 403 they're not going to be able to crawl.

            zita.vn 1 Reply Last reply Reply Quote 1
            • KristinaKeyser
              KristinaKeyser last edited by

              Hi there! Kristina from Moz's Help Team here.

              When our crawler attempts to access your site, we are running into this error: http://screencast.com/t/CmprKKb7a "403 Forbidden".

              This normally means that you're blocking us on a server level. When I ran your site in a header checker I was able to see a 200, but when I curled your site from our servers it returned a 403.

              The bummer here is that I can't do anything on the support side to fix this. 😞

              This 403 error is coming directly from your server, so at this point the best thing to do will be to send this information to your web dev. They'll need to make sure our crawler (rogerbot) is whitelisted, so our tools can start working hard for you.

              As always, you can contact our team on issues like this in the future by emailing help@moz.com, but hopefully this helps point you in the right direction towards a solution here.

              Thank you,
              -Kristina

              zita.vn 1 Reply Last reply Reply Quote 0
              • zita.vn
                zita.vn @KristinaKeyser last edited by

                Hi Kristina,

                Thank you for your answer.

                Currently to protect our site from being hacked we need your user agent to add it into our whitelisted, so you will be able to crawl our website.

                Regards,

                Phuong.

                vcj 1 Reply Last reply Reply Quote 0
                • vcj
                  vcj @zita.vn last edited by

                  The useragent is: rogerbot

                  1 Reply Last reply Reply Quote 1
                  • zita.vn
                    zita.vn @vcj last edited by

                    Hi there,

                    I have been checking my site for a week, but I havent seen any new updates about my site in backlink. Im sure that since last week there are a few sites pointing a link to my website but I havent found it on Moz.

                    Please explain it for me.

                    Tks a lot,

                    1 Reply Last reply Reply Quote 0
                    • 1 / 1
                    • First post
                      Last post
                    • Moz Crawl only crawling the top level page (1 page)
                      tawnycase
                      tawnycase
                      0
                      2
                      198

                    • Why is Moz Crawling More Pages Than My Site Actually Has?
                      tawnycase
                      tawnycase
                      0
                      2
                      60

                    • Page Authority drop to Zero with new Moz crawl
                      Hurf
                      Hurf
                      0
                      2
                      360

                    • Duplicate page found with MOZ crawl test?
                      rhondafranklin
                      rhondafranklin
                      0
                      4
                      292

                    • Why does the moz crawl test lists page twice?
                      iSTORM-New-Media
                      iSTORM-New-Media
                      0
                      3
                      224

                    • Moz Crawl Test says pages have no internal links
                      ChiarynMiranda
                      ChiarynMiranda
                      0
                      4
                      331

                    • Optimise your Pages in Moz Crawl - where do the keywords come from?
                      DavidLee
                      DavidLee
                      0
                      4
                      366

                    • How do you stop Moz crawling a page?
                      signcreator
                      signcreator
                      0
                      6
                      2.5k

                    Get started with Moz Pro!

                    Unlock the power of advanced SEO tools and data-driven insights.

                    Start my free trial
                    Products
                    • Moz Pro
                    • Moz Local
                    • Moz API
                    • Moz Data
                    • STAT
                    • Product Updates
                    Moz Solutions
                    • SMB Solutions
                    • Agency Solutions
                    • Enterprise Solutions
                    • Digital Marketers
                    Free SEO Tools
                    • Domain Authority Checker
                    • Link Explorer
                    • Keyword Explorer
                    • Competitive Research
                    • Brand Authority Checker
                    • Local Citation Checker
                    • MozBar Extension
                    • MozCast
                    Resources
                    • Blog
                    • SEO Learning Center
                    • Help Hub
                    • Beginner's Guide to SEO
                    • How-to Guides
                    • Moz Academy
                    • API Docs
                    About Moz
                    • About
                    • Team
                    • Careers
                    • Contact
                    Why Moz
                    • Case Studies
                    • Testimonials
                    Get Involved
                    • Become an Affiliate
                    • MozCon
                    • Webinars
                    • Practical Marketer Series
                    • MozPod
                    Connect with us

                    Contact the Help team

                    Join our newsletter
                    Moz logo
                    © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                    • Accessibility
                    • Terms of Use
                    • Privacy