The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Is it possible to exclude pages from Crawl Diagnostic?

    Is it possible to exclude pages from Crawl Diagnostic?

    Moz Tools
    2 2 425
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • wfernley
      wfernley last edited by

      I like the crawl diagnostic but it shows many errors due to a forum that I have. I don't care about the SEO value of this forum and would like to exclude any pages in the /forum/ directory. Is it possible to add exclusions to the crawl diagnostic tool?

      1 Reply Last reply Reply Quote 1
      • GCSMasone
        GCSMasone last edited by

        You can exclude those by placing them in a robot.txt file. They crawler will obey that and not follow. Learn about Crawl Diagnostics here.

        Best of luck in your work!

        Brandon

        1 Reply Last reply Reply Quote 1
        • 1 / 1
        • First post
          Last post
        • Crawl Diagnostics saids a page is linking but I can't find the link on the page.
          DirkC
          DirkC
          0
          4
          128

        • Crawl Diagnostics 403 on home page...
          ChiarynMiranda
          ChiarynMiranda
          0
          9
          420

        • I've got quite a few "Duplicate Page Title" Errors in my Crawl Diagnostics for my Wordpress Blog
          Devanur-Rafi
          Devanur-Rafi
          0
          2
          292

        • Is it possible to override the 10k pages crawl limit on PRO?
          Richline_Digital
          Richline_Digital
          0
          4
          503

        • Too Many On-Page Links: Crawl Diag vs On-Page
          Dryope
          Dryope
          0
          3
          481

        • Dynamic URL pages in Crawl Diagnostics
          cmaseattle
          cmaseattle
          0
          4
          681

        • Crawl Diagnostics and missing meta tags on noindex blog pages
          ShaMenz
          ShaMenz
          0
          2
          835

        • My website has 18500 pages but my SEO MOZ campaign is limited to a 10,000 page crawl. How can I get the other 8500 pages crawled? Can I use one of my 3 spare campaigns?
          kenneth_martin
          kenneth_martin
          0
          5
          1.0k

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy