The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. How to block my web from SeoMOz Crawler

    How to block my web from SeoMOz Crawler

    Technical SEO Issues
    2 2 821
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Gomu2
      Gomu2 last edited by

      Hi,

      I want to block mysite.com/forum from SeoMoz Crawler, how do i do that? i just want to block the forum and let the other part of my site still crawlable by seomoz. Thanks

      Regards

      1 Reply Last reply Reply Quote 0
      • TomRayner
        TomRayner last edited by

        Howdy

        You'll need to block the SEOMoz crawler in your robots.txt file.  Edit the file and insert this bit of code:

        User-Agent: rogerbot Allow: /
        Disallow: /forum/

        If that doesn't work, you might want to add a Wildcard too, making it:

        Disallow: *****/forum/

        If you're not too sure about editing the robots.txt file, then it might be an idea to check the SEOMoz guide: http://www.seomoz.org/learn-seo/robotstxt - if you're really unsure, it's best to get someone else to do it for you, as you don't want to block all of your site to all of the crawlers

        Hope this helps

        1 Reply Last reply Reply Quote 2
        • 1 / 1
        • First post
          Last post
        • Need Third Party Input. Our Web host blocked all bots including Google and myself because they believe SEO is slowing down their server.
          KeriMorgret
          KeriMorgret
          1
          9
          314

        • Blocking subdomains without blocking sites...
          OlegKorneitchouk
          OlegKorneitchouk
          1
          6
          145

        • How can I block incoming links from a bad web site ?
          Kurt_Steinbrueck
          Kurt_Steinbrueck
          0
          4
          8.1k

        • While SEOMoz currently can tell us the number of linking c-blocks, can SEOMoz tell us what the specific c-blocks are?
          CMC-SD
          CMC-SD
          0
          2
          292

        • SeoMoz crawler giving false positives?
          kenneth_martin
          kenneth_martin
          0
          9
          599

        • Warnings for blocked by blocked by meta-robots/meta robots Nofollow...how to resolve?
          Cyrus-Shepard
          Cyrus-Shepard
          0
          3
          415

        • Mobile SEO or Block Crawlers?
          Michael_Martin
          Michael_Martin
          0
          2
          981

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy