The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Googlebot take 5 times longer to crawl each page

    Googlebot take 5 times longer to crawl each page

    Technical SEO Issues
    5 2 314
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • spes123
      spes123 last edited by

      Hello All

      From about mid September my GWMT has show that the average  time to crawl a page on my site has shot up from an average of 130ms to an average of 700ms and peaks at 4000ms.

      I have checked my server error logs and found nothing there, I have checked with the hosting comapny and there are no issues with the server or other sites on the same server.

      Two weeks after this my ranking fell by about 950 places for most of my keywords etc.I am really just trying to eliminate this as a possible cause, of these ranking drops. Or was it the Pand/ EMD algo that has done it.

      Many Thanks

      Si

      1 Reply Last reply Reply Quote 0
      • eyepaq
        eyepaq last edited by

        Hi,

        As far as I know a low crawl rate won't end up with bad rankings but bad rankings will end up with a lower crawl rate.

        If you are sure and I mean really sure you don't have any technical issues on your side that will influence the crawl rate and possibile also rankings then you should take in consideration that maybe you do actually have a -950 filter that is causing your rankings to drop, google dosen't consider your site an authority and for thsi reason it won't crawl your site often or as often as it used to do it.

        Can you share the url of the site ? Just to have a look and see if at a first glance there are any obvios reason for google to dislike your site.

        Cheers !

        spes123 1 Reply Last reply Reply Quote 0
        • spes123
          spes123 @eyepaq last edited by

          Hi

          I am as sure as I can be but not being a full expert on these things I may have missed something technical.

          I have be making changes to the site since mainly on the css layout.

          The site is www.growingyourownveg.com

          Thanks

          eyepaq spes123 2 Replies Last reply Reply Quote 0
          • eyepaq
            eyepaq @spes123 last edited by

            Hi.

            Based on the site size  (number of pages) and format (code, elements and structure) and two speed test I just run on it and a trace-route (from Austria) looks like you don't have any issues with it from a technical point of view.

            One thing you need to check is still possibile is the time spent downloding a page graph (the third one) from within GWMT. Did this spiked up in the same time when crawl pages went down ?

            A few other questions you should consider:

            • did you do any changes - especially structure changes around the same time you've notice the issues ?

            • are there any public google updates in the same timeframe with those changes that  you've notice ?( you can check them here: http://www.seomoz.org/google-algorithm-change )

            • is your content duplicate ? (with external sources I mean - not internally)

            Please don't get me wrong - i would be ok with the format of the site if it will be very old - before 2000. But the domain is from 2008 - you should get on track with new trends as far as layout, content format and web site format in general.

            Hope it helps.

            1 Reply Last reply Reply Quote 0
            • spes123
              spes123 @spes123 last edited by

              Thank you for having a look

              I made no strcutural changes around the time of the issues starting.

              On the third graph in GWMT yes there is was a spike on the time spent downloading at it is still a lot higher than previously. I have add an image of it below.

              There were two google update about two weeks later the latest Panda and the new EMD.

              Most of the content has been written by myself from my own experience etc. There are some pages that I am in the process of removing / changing that are the same as other sites.

              Until 4 months  ago the layout was in fixed size nested tables etc, I am just about getting my head around CSS etc., to try and drag it in the 21st century.

              ALKBp.png ALKBp.png

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Very wierd pages. 2900 403 errors in page crawl for a site that only has 140 pages.
                H.M.N.
                H.M.N.
                0
                6
                64

              • Site Crawl -> Duplicate Page Content -> Same pages showing up with duplicates that are not
                davebuts
                davebuts
                0
                2
                72

              • How come only 2 pages of my 16 page infographic are being crawled by Moz?
                EricksonCoaching
                EricksonCoaching
                0
                7
                91

              • If the order of products on a page changes each time the page is loaded, does this have a negative effect on the SEO of those pages?
                Kurt_Steinbrueck
                Kurt_Steinbrueck
                2
                4
                119

              • I have 3500 pages crawled by Google, - why is SEOMOZ only able to crawl 400 of these ?
                Abe_Schmidt
                Abe_Schmidt
                0
                2
                95

              • Hello every one please give me idea about offpage techniques for keyword ranking for first page on top 5 all search engine ,and it must be durable for long time. If i will not use on page optimization?
                MarkScully
                MarkScully
                0
                9
                499

              • SEOMoz Crawl Diagnostic indicates duplicate page content for home page?
                Linesides
                Linesides
                0
                3
                607

              • Crawl report showing only 1 crawled page
                Mikpam
                Mikpam
                0
                4
                913

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy