The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Stop Google crawling a site at set times

    Stop Google crawling a site at set times

    Intermediate & Advanced SEO
    5 2 164
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • ske11
      ske11 last edited by

      Hi All

      I know I can use robots.txt to block Google from pages on my site but is there a way to stop Google crawling my site at set times of the day? Or to request that they crawl at other times?

      Thanks

      Sean

      1 Reply Last reply Reply Quote 0
      • KeriMorgret
        KeriMorgret last edited by

        I know Bing has a way to do that, but I haven't heard of Google releasing a similar tool.

        ske11 1 Reply Last reply Reply Quote 0
        • ske11
          ske11 @KeriMorgret last edited by

          Thanks for that Keri, do you know what Bing calls that option , it might give me a starting point for a Google search

          Sean

          KeriMorgret ske11 2 Replies Last reply Reply Quote 0
          • KeriMorgret
            KeriMorgret @ske11 last edited by

            Here's a link to Bing's help page on the tool. http://www.bing.com/webmaster/help/crawl-control-55a30302.

            1 Reply Last reply Reply Quote 1
            • ske11
              ske11 @ske11 last edited by

              Thanks Keri, this gives me a place to start

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • Google Mobile site crawl returns poorer results on 100% responsive site
                WebQuest
                WebQuest
                0
                4
                37

              • Trying to get Google to stop indexing an old site!
                Martijn_Scheijbeler
                Martijn_Scheijbeler
                0
                6
                103

              • Set Robots.txt file to crawl my website at specific times
                Tenlo
                Tenlo
                0
                2
                50

              • Google crawling 200 page site thousands of times/day. Why?
                brettmandoes
                brettmandoes
                0
                7
                90

              • Why is my site not getting crawled by google?
                DonnaDuncan
                DonnaDuncan
                0
                4
                162

              • Should I let Google crawl my production server if the site is still under development?
                DoItHappy
                DoItHappy
                0
                4
                291

              • Google crawling the demo site, robots exclusion already placed
                donford
                donford
                0
                2
                541

              • Working out exactly how Google is crawling my site if I have loooots of pages
                wazza1985
                wazza1985
                0
                3
                875

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy