The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Stop Google crawling a site at set times

    Stop Google crawling a site at set times

    Intermediate & Advanced SEO
    5 2 164
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ske11
      ske11 last edited by

      Hi All

      I know I can use robots.txt to block Google from pages on my site but is there a way to stop Google crawling my site at set times of the day? Or to request that they crawl at other times?

      Thanks

      Sean

      1 Reply Last reply Reply Quote 0
      • KeriMorgret
        KeriMorgret last edited by

        I know Bing has a way to do that, but I haven't heard of Google releasing a similar tool.

        ske11 1 Reply Last reply Reply Quote 0
        • ske11
          ske11 @KeriMorgret last edited by

          Thanks for that Keri, do you know what Bing calls that option , it might give me a starting point for a Google search

          Sean

          KeriMorgret ske11 2 Replies Last reply Reply Quote 0
          • KeriMorgret
            KeriMorgret @ske11 last edited by

            Here's a link to Bing's help page on the tool. http://www.bing.com/webmaster/help/crawl-control-55a30302.

            1 Reply Last reply Reply Quote 1
            • ske11
              ske11 @ske11 last edited by

              Thanks Keri, this gives me a place to start

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Trying to get Google to stop indexing an old site!
                Martijn_Scheijbeler
                Martijn_Scheijbeler
                0
                6
                103

              • Google crawling 200 page site thousands of times/day. Why?
                brettmandoes
                brettmandoes
                0
                7
                90

              • My site shows 503 error to Google bot, but can see the site fine. Not indexing in Google. Help
                Everett
                Everett
                0
                3
                733

              • How to make Google index your site? (Blocked with robots.txt for a long time)
                SanjidaKazi
                SanjidaKazi
                0
                3
                128

              • After Receiving a "Googlebot can't access your site" would this stop your site from being crawled?
                evolvingSEO
                evolvingSEO
                0
                4
                394

              • How to stop Google crawling after 301 redirect?
                irvingw
                irvingw
                0
                2
                997

              • Google crawl errors & serps [adult site]
                Ramp
                Ramp
                0
                6
                799

              • How to Set Custom Crawl Rate in Google Webmaster Tools?
                CommercePundit
                CommercePundit
                0
                3
                798

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy