The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Getting Started
    4. How do i stop Moz from indexing my dev site?

    How do i stop Moz from indexing my dev site?

    Getting Started
    2 2 113
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Tholomew1
      Tholomew1 last edited by

      Hey,

      I am new to MOZ, and I am seeing in my page reports that I have duplicate content, but this duplicate content is mostly my development site. dev.domain.com. I have this blocked in robots.txt for google.

      How do i stop MOZ including it in its reports?

      Cheers everyone.
      Bart

      1 Reply Last reply Reply Quote 0
      • Alick300
        Alick300 last edited by

        Hi,

        To stop moz crawler you have to mention below text in robots.txt for dev.domain.com

        User-Agent: rogerbot Disallow: /

        Hope this helps.

        Thanks

        1 Reply Last reply Reply Quote 2
        • 1 / 1
        • First post
          Last post
        • Moz not able to crawl our site - any advice?
          Salience_Search_Marketing
          Salience_Search_Marketing
          1
          3
          81

        • Moz can't crawl my site.
          eli.myers
          eli.myers
          0
          2
          118

        • Why does the MOZ bar not work on SERPs with site links?
          samantha.chapman
          samantha.chapman
          0
          2
          73

        • Duplicate Content after Moz Site Audit
          ATP
          ATP
          0
          6
          205

        • What Moz tool is best to find reasons google has not spidered by site
          Sean_Peerenboom
          Sean_Peerenboom
          0
          4
          137

        • I could not find a support number for Moz on your site
          KeriMorgret
          KeriMorgret
          1
          10
          195

        • What is the "other" group in traffic to your site on the moz dashboard ?
          Abe_Schmidt
          Abe_Schmidt
          0
          4
          127

        • How do get Moz to spider a Development site PRE LAUNCH?
          MattRoney
          MattRoney
          0
          3
          146

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy