The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Getting Started
    4. Crawling Website

    Crawling Website

    Getting Started
    5 3 119
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Leoni
      Leoni last edited by

      How often does Moz crawl the website? I have a lot of high priority crawl issues but my web development team says that it fixed them. However, Moz still shows me that none of the errors are fixed.

      1 Reply Last reply Reply Quote 0
      • irvingw
        irvingw last edited by

        Please see this discussion to help you out.

        http://moz.com/community/q/how-do-i-force-a-crawl

        1 Reply Last reply Reply Quote 0
        • KeriMorgret
          KeriMorgret last edited by

          We crawl on a weekly basis. You can force a crawl for up to 3000 URLs at the crawl test at https://moz.com/researchtools/crawl-test.

          Leoni 1 Reply Last reply Reply Quote 0
          • Leoni
            Leoni @KeriMorgret last edited by

            So will it update automatically in my overview as well? When will it be crawled on a weekly basis, in the beginning of the week or end of the week? How do I know if my website has been crawled yet?

            KeriMorgret 1 Reply Last reply Reply Quote 0
            • KeriMorgret
              KeriMorgret @Leoni last edited by

              The crawl test will show the results in a CSV file, but not update on your dashboard. It is a way to spot-check and see if the changes have been made.

              When you go to your crawl diagnostics page, it will tell you the date of the last crawl. We crawl on the same day each week.

              1 Reply Last reply Reply Quote 1
              • 1 / 1
              • First post
                Last post
              • Risk of moz on website
                eli.myers
                eli.myers
                0
                2
                35

              • Can't Crawl Site - but deducting crawls.
                dave.kudera
                dave.kudera
                1
                3
                52

              • Crawling issue
                eli.myers
                eli.myers
                0
                2
                52

              • When I crawl my site On Moz it says it can't access the robots.txt file, but crawl is fine on SEM Rush - Anyone know any reason for this?
                meghanpahinui
                meghanpahinui
                0
                3
                60

              • Crawl rate
                FuelDump
                FuelDump
                0
                6
                324

              • My website does not allow all crawler to crawl, Now my question is that whether i need to give permission to moz crawler if yes then whaat is moz bot name?
                MonicaOConnor
                MonicaOConnor
                0
                3
                794

              • Website errors?
                Essential-Pest
                Essential-Pest
                1
                3
                106

              • After fixing Crawl Errors, how long does it take to for Moz or Google to re-crawl a website?
                DougRoberts
                DougRoberts
                0
                2
                650

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy