The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz News
    4. Can i exclude paths from my crawl?

    Can i exclude paths from my crawl?

    Moz News
    2 2 438
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • royboy
      royboy last edited by

      My site is setup like this mysite.com with mysite.com/blog

      The blog portion is overwhelming my SEO efforts - I'd like to exclude that from seomoz reporting?  is there anyway to do that in the campaign manager? or will I have to modify my robots.txt to inform roger not to crawl there?

      1 Reply Last reply Reply Quote 0
      • Anthony_NorthSEO
        Anthony_NorthSEO last edited by

        Here's SEOMoz's Article on the subject, authored by Rogerbot himself:

        http://www.seomoz.org/dp/rogerbot

        You just need to add this to your robots.txt:

        <code>User-agent: rogerbot 
        Disallow: /blog</code>
        

        Should do the trick.

        1 Reply Last reply Reply Quote 2
        • 1 / 1
        • First post
          Last post
        • How can i increase DA of my site?
          0
          3
          61

        • I have assigned a Moz Pro seat to someone but they can't access a campaign's rankings report?
          0
          1
          31

        • Please who can help me rank my website www.shopdocuments.com ?
          jasongmcmahon
          jasongmcmahon
          1
          2
          92

        • Non SEO Question, but hopefully some from Moz / Seattle can help me
          Andy-Halliday
          Andy-Halliday
          0
          4
          289

        • How soon can we expect MozCon 2013 videos?
          KeriMorgret
          KeriMorgret
          4
          5
          254

        • Can I chat with seomoz.org?
          Christy-Correll
          Christy-Correll
          0
          8
          1.2k

        • I find these errors in seomoz how can i remove it, wordpress blog
          TammyWood
          TammyWood
          0
          3
          367

        • What can I do different to increase my keyword ranking
          OgyDog
          OgyDog
          0
          11
          1.8k

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy