The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Intermediate & Advanced SEO
    4. Block web archieve/way back machine

    Block web archieve/way back machine

    Intermediate & Advanced SEO
    2 2 342
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Mikey008
      Mikey008 last edited by

      Hi i want to block web archive/wayback machine from indexing my site and creating a record of it on their database.

      Any ideas on how to do this?

      Cheers,
      Superpak

      1 Reply Last reply Reply Quote 2
      • davebuts
        davebuts last edited by

        You can block Wayback Machine from crawling and creating a record of your site by adding the following to your Robots.txt file:

        User-agent: ia_archiver
        Disallow: /

        This will not only stop new records from being created but also stop people viewing what had previously been indexed by Wayback Machine.

        More information about this can be found here: https://archive.org/about/exclude.php

        1 Reply Last reply Reply Quote 9
        • 1 / 1
        • First post
          Last post
        • I have a metadata issue. My site crawl is coming back with missing descriptions, but all of the pages look like site tags (i.e. /blog/?_sft_tag=call-routing)
          Rajesh.Prajapati
          Rajesh.Prajapati
          0
          2
          43

        • Is Link equity / Link Juice lost to a blocked URL in the same way that it is lost to nofollow link
          NickSamuel
          NickSamuel
          0
          2
          33

        • Visibility for https://goo.gl/gJH7eh
          CraigBradford
          CraigBradford
          0
          7
          197

        • Domain Authority... http://www.domain.com/ vs. http://domain.com vs. http://domain.com/
          MattAntonino
          MattAntonino
          0
          4
          136

        • Https://www.mywebsite.com/blog/tag/wolf/ setting tag pages as blog corner stone article?
          AlirezaHamidian
          AlirezaHamidian
          0
          6
          135

        • URL Redirect: http://www.example.net/ vs. http://www.example.net
          dotfly
          dotfly
          0
          2
          705

        • Using a 302 re-direct from http://www to https://www to secure customer data
          etruvian
          etruvian
          0
          6
          123

        • Should we block urls like this - domainname/shop/leather-chairs.html?brand=244&cat=16&dir=ascℴ=price&price=1 within the robots.txt?
          sferrino
          sferrino
          0
          2
          864

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy