The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. I am trying to block robots from indexing parts of my site..

    I am trying to block robots from indexing parts of my site..

    Technical SEO Issues
    3 2 374
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Smurkcreative
      Smurkcreative last edited by

      I have a few websites that I mocked up for clients to check out my work and get a feel for the style I produce but I don't want them indexed as they have lore ipsum place holder text and not really optimized... I am in the process of optimizing them but for the time being I would like to block them. Most of my warnings and errors on my seomoz dashboard are from these sites and I was going to upload the folioing to the robot.txt file but I want to make sure this is correct:

      User-agent: *

      Disallow: /salondemo/

      Disallow: /salondemo3/

      Disallow: /cafedemo/

      Disallow: /portfolio1/

      Disallow: /portfolio2/

      Disallow: /portfolio3/

      Disallow: /salondemo2/

      is this all i need to do?

      Thanks

      Donny

      1 Reply Last reply Reply Quote 0
      • AlanBleiweiss
        AlanBleiweiss last edited by

        this is the correct approach when using the robots.txt method of blocking.  Be aware however, that the only secure way to ensure 100% that such locations are not indexed is to put them behind a password protected gate-way.  I always recommend to design agencies that there be a simple single log-in screen between the front end and design folders.  This can be as complex as unique UIDs and Passwords for every client, or a single shared login, if all you want to do is bar search engines from seeing the content.

        1 Reply Last reply Reply Quote 1
        • Smurkcreative
          Smurkcreative last edited by

          Thanks

          1 Reply Last reply Reply Quote 0
          • 1 / 1
          • First post
            Last post
          • Should you use robots.txt for pages within your site which do not have high quality content or are not contributing a great deal so when Google crawls your site the best performing content has a higher chance of being indexed?
            Jacksons_Fencing
            Jacksons_Fencing
            0
            5
            44

          • Google is indexing blocked content in robots.txt
            bjs2010
            bjs2010
            0
            5
            147

          • Google index dymamic webpages after block in robots.txt...
            Dr-Pete
            Dr-Pete
            0
            6
            247

          • My beta site (beta.website.com) has been inadvertently indexed. Its cached pages are taking traffic away from our real website (website.com). Should I just "NO INDEX" the entire beta site and if so, what's the best way to do this? Please advise.
            Vuly
            Vuly
            0
            5
            1.6k

          • Accidentally blocked whole site with robots.txt and struggling to regain rankings.
            SebastianCowie
            SebastianCowie
            0
            5
            998

          • Site blocked by robots.txt and 301 redirected still in SERPs
            OlegKorneitchouk
            OlegKorneitchouk
            0
            2
            621

          • Robots.txt blocking site or not?
            RyanKent
            RyanKent
            0
            2
            445

          • Site not being Indexed that fast anymore, Is something wrong with this Robots.txt
            oznappies
            oznappies
            0
            4
            828

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy