The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Site maintenance and crawling

    Site maintenance and crawling

    Technical SEO Issues
    3 2 451
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Daylan
      Daylan last edited by

      Hey all,

      Rarely, but sometimes we require to take down our site for server maintenance, upgrades or various other system/network reasons. More often than not these downtimes are avoidable and we can redirect or eliminate the client side downtime.

      We have a 'down for maintenance - be back soon' page that is client facing. ANd outages are often no more than an hour tops.

      My question is, if the site is crawled by Bing/Google at the time of site being down, what is the best way of ensuring the indexed links are not refreshed with this maintenance content? (ie: this is what the pages look like now, so this is what the SE will index).

      I was thinking that add a no crawl to the robots.txt for the period of downtime and remove it once back up, but will this potentially affect results as well?

      1 Reply Last reply Reply Quote 1
      • KeriMorgret
        KeriMorgret last edited by

        We just had a great YouMoz post on this (that was promoted to the main blog) that might help for your next maintenance window. http://www.seomoz.org/blog/how-to-handle-downtime-during-site-maintenance

        Daylan 1 Reply Last reply Reply Quote 1
        • Daylan
          Daylan @KeriMorgret last edited by

          Great post. Thanks very much.

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Should you use robots.txt for pages within your site which do not have high quality content or are not contributing a great deal so when Google crawls your site the best performing content has a higher chance of being indexed?
            Jacksons_Fencing
            Jacksons_Fencing
            0
            5
            44

          • Why can no tool crawl this site?
            spencerhjustice
            spencerhjustice
            0
            9
            205

          • Seomoz Can not Crawl My Site
            AndreVanKets
            AndreVanKets
            0
            2
            112

          • How Often is Site Crawled
            Matt-Williamson
            Matt-Williamson
            0
            5
            1.8k

          • Is any know if seomoz update for site crawl.
            KeriMorgret
            KeriMorgret
            0
            2
            310

          • Site Crawl
            Sean_Dawes
            Sean_Dawes
            0
            2
            414

          • How do you diagnose if on your site is only 50% crawled?
            Nightwing
            Nightwing
            0
            3
            326

          • Recently revamped site structure - now not even ranking for brand name, but lots of content - what happened? (Yup, the site has been crawled a few times since) Any ideas? Did I make a classic mistake? Any advise appreciated :)
            AlanBleiweiss
            AlanBleiweiss
            0
            9
            844

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy