The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Google crawl rate dropped after we activated CloudFront

    Google crawl rate dropped after we activated CloudFront

    Technical SEO Issues
    2 2 848
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Ludde
      Ludde last edited by

      Hello!

      Previously we've been using Amazon CloudFront for our static content (js, css etc). But to be able to reduce load on our origin servers and to be able to give our international users a good user experience we decided to deliver a couple of our sites through CloudFront.

      We noticed very nice drops in page load time, but when checking Google webmaster tools we noticed that all CloudFront-activated sites got a huge drop in pages crawled per day (from avg ~3500 to ~150). Also one of the sites have issues with the Google sitemaps (just marked as "Pending" in GWT) and no new pages or updated pages seems to be updated in the Google SERP. The rest of the sites gets some updates on the Google SERP, but very few compared to before CloudFront activation.

      Is there anybody here who have experience in full site delivery through CloudFront (or other CDNs) and effects on SEO/Google? Would be very glad for any insights or suggestions. The risk is that we need to remove CloudFront if this just continues.

      1 Reply Last reply Reply Quote 0
      • EricaMcGillivray
        EricaMcGillivray last edited by

        My very best educated guess (after talking with dev friends here at Moz) is that CloudFront is supposed to be used for long-term storage. This means the content isn't expected to change frequently. This is great for images and other static things. However, this is also a signal to Google not to crawl your site as often as it won't be changing that often.

        1 Reply Last reply Reply Quote 1
        • 1 / 1
        • First post
          Last post
        • Web Page Dropped Out of Google?
          MattAntonino
          MattAntonino
          0
          4
          119

        • Can Google Crawl This Page?
          N1ghteyes
          N1ghteyes
          0
          4
          263

        • 404s effecting crawl rate?
          DennisSeymour
          DennisSeymour
          0
          4
          126

        • Crawling issues in google
          CMTM
          CMTM
          0
          4
          258

        • My Website is not being crawled by Google
          MarieHaynes
          MarieHaynes
          0
          16
          2.0k

        • Suggested crawl rate in google webmaster tools?
          BailHotline
          BailHotline
          0
          4
          1.2k

        • How to increase the crawl rate?
          greyniumseo
          greyniumseo
          0
          3
          675

        • Blocking Google from Crawling Parameters
          wissamdandan
          wissamdandan
          0
          7
          1.1k

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy