The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. 404s effecting crawl rate?

    404s effecting crawl rate?

    Technical SEO Issues
    4 4 126
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • JoshKimber
      JoshKimber last edited by

      We made a change to our site where we all of a sudden we are creating a large number of 404 pages.  Is this effecting the crawl/indexing rate?

      Currently we've submitted 3.4 million pages, have over 834K indexed but have over and 330K pages not found.  Since the large increase in 404s we've noticed a decrease in pages crawled per day.  I found this Q & A in Webmasters (http://googlewebmastercentral.blogspot.com/2011/05/do-404s-hurt-my-site.html) but it seems like the 404s should not have an effect.  Is this article out of date?

      What do you think fellow Moz-ers? Is this a problem?

      1 Reply Last reply Reply Quote 0
      • kayintveen_MD
        kayintveen_MD last edited by

        Yes i've seen this numerous of times. Is it just 404's are are there also things like DNS playing along?
        But if the amount of 404's jump up really high then for sure google turns down the speed. i guess this gives you some air to fix it in time.

        1 Reply Last reply Reply Quote 1
        • eyepaq
          eyepaq last edited by

          That article you mention is very up to date. 🙂  but if you got "hit" by Google bot several times a day for those pages that now you return a 404 response code you will see a decrease in pages crawled per day since once Google sees a 404 response code it will not visit / hit that page that often aftre that...

          1 Reply Last reply Reply Quote 1
          • DennisSeymour
            DennisSeymour last edited by

            It's not a problem, just fix those as soon as you can. And yes, it does affect crawl rate from what I've seen.

            1 Reply Last reply Reply Quote 1
            • 1 / 1
            • First post
              Last post
            • Internal link is creating duplicate content issues and generating 404s from website crawl.
              VacuumCleanersBest
              VacuumCleanersBest
              0
              2
              49

            • Google Crawling Issues! How Can I Get Google to Crawl My Website Regularly?
              libero_net
              libero_net
              1
              8
              142

            • Crawl issue
              Mrlocicero
              Mrlocicero
              0
              3
              202

            • Have a client that migrated their site; went live with noindex/nofollow and for last two SEOMoz crawls only getting one page crawled. In contrast, G.A. is crawling all pages. Just wait?
              Nobody1560986989723
              Nobody1560986989723
              0
              5
              422

            • Googlebot Crawl Rate causing site slowdown
              Rich_A
              Rich_A
              0
              8
              6.1k

            • Suggested crawl rate in google webmaster tools?
              BailHotline
              BailHotline
              0
              4
              1.2k

            • Linklicious and Crawl rates
              jenadams
              jenadams
              0
              2
              545

            • Google crawl rate almost zero since re-launch, organic search up 50% though!
              ShaMenz
              ShaMenz
              0
              4
              819

            Get started with Moz Pro!

            Unlock the power of advanced SEO tools and data-driven insights.

            Start my free trial
            Products
            • Moz Pro
            • Moz Local
            • Moz API
            • Moz Data
            • STAT
            • Product Updates
            Moz Solutions
            • SMB Solutions
            • Agency Solutions
            • Enterprise Solutions
            • Digital Marketers
            Free SEO Tools
            • Domain Authority Checker
            • Link Explorer
            • Keyword Explorer
            • Competitive Research
            • Brand Authority Checker
            • Local Citation Checker
            • MozBar Extension
            • MozCast
            Resources
            • Blog
            • SEO Learning Center
            • Help Hub
            • Beginner's Guide to SEO
            • How-to Guides
            • Moz Academy
            • API Docs
            About Moz
            • About
            • Team
            • Careers
            • Contact
            Why Moz
            • Case Studies
            • Testimonials
            Get Involved
            • Become an Affiliate
            • MozCon
            • Webinars
            • Practical Marketer Series
            • MozPod
            Connect with us

            Contact the Help team

            Join our newsletter
            Moz logo
            © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
            • Accessibility
            • Terms of Use
            • Privacy