The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Block or remove pages using a robots.txt

    Block or remove pages using a robots.txt

    Technical SEO Issues
    2 2 422
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • semer
      semer last edited by

      I want to use robots.txt to prevent googlebot access the specific folder on the server,

      Please tell me if the syntax below is correct

      User-Agent: Googlebot Disallow: /folder/

      I want to use robots.txt to prevent google image index the images of  my website  ,

      Please tell me if the syntax below is correct

      User-agent: Googlebot-Image

      Disallow: /

      1 Reply Last reply Reply Quote 0
      • OlegKorneitchouk
        OlegKorneitchouk last edited by

        Yes, that is correct.

        Once you have it set up, you can test what pages are blocked in your GWT account. Health > Blocked URLs > select which bot you want to test at the bottom of the page.

        Resources:

        Google's bots - http://support.google.com/webmasters/bin/answer.py?hl=en&answer=1061943

        WMT Blocked URL testing - https://www.google.com/webmasters/tools/robots-analysis

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Can I Block https URLs using Host directive in robots.txt?
          LoganRay
          LoganRay
          0
          4
          760

        • Should I block Map pages with robots.txt?
          imaginex
          imaginex
          0
          3
          73

        • Using the Google Remove URL Tool to remove https pages
          sparrowdog
          sparrowdog
          1
          3
          5.9k

        • The use of robots.txt
          ICON_Malta
          ICON_Malta
          0
          3
          84

        • Do you get credit for an external link that points to a page that's being blocked by robots.txt
          ShaMenz
          ShaMenz
          0
          2
          390

        • Robots.txt blocking site or not?
          RyanKent
          RyanKent
          0
          2
          445

        • Blocking other engines in robots.txt
          RyanKent
          RyanKent
          0
          2
          581

        • Search Engine Blocked by Robot Txt warnings for Filter Search result pages--Why?
          languedoc
          languedoc
          0
          5
          915

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy