The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Moz Tools
    4. Hoe to crawl specific subfolders

    Hoe to crawl specific subfolders

    Moz Tools
    2 2 264
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • gofluent
      gofluent last edited by

      I tried to create a campaign to crawl the subfolders of my site, but it stops at just 1 folder. Basically what I want to do is crawl everything after folder1:

      www.domain.com/web/folder1/*

      I tried to create 2 campaigns:

      Subfolder Campaign 1: www.domain.com/web/folder1/*
      Subfolder Campaign 2: www.domain.com/web/folder1/

      In both cases, it did not crawl and folders after the last /. Can you help me ?

      1 Reply Last reply Reply Quote 0
      • Kara.Wallace
        Kara.Wallace last edited by

        According to Moz Help, "The PRO Web App crawls, grades and checks rankings only for pages found on that domain or subfolder. When tracking a subfolder, we will only crawl pages that exist in that specific folder. We will not crawl links that go to other folders."

        Hope this helps.

        Mike

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Block Moz (or any other robot) from crawling pages with specific URLs
          Blacktie
          Blacktie
          0
          3
          844

        • Crawl diagnostics up to date after Magento ecommerce site crawl?
          Whebb
          Whebb
          0
          2
          218

        • How do I exclude my blog subfolder from being crawled with my main domain (www.) folder?
          Svenne
          Svenne
          0
          7
          329

        • How often does seomoz crawl the site? Can you force a crawl at a specific time ?
          stewbuch1872
          stewbuch1872
          0
          6
          188

        • Crawl Disgnosis only crawling 250 pages not 10,000
          kenneth_martin
          kenneth_martin
          0
          7
          409

        • Crawl test tool from SEOmoz - which URLs does it actually crawl?
          ShaMenz
          ShaMenz
          0
          4
          787

        • Only 1 page is being crawled by SEOmoz for the last 2 crawls
          KeriMorgret
          KeriMorgret
          0
          6
          863

        • How can I change (specifically, decrease) the reporting/crawling frequency of the keyword ranking?
          MackenzieFogelson
          MackenzieFogelson
          0
          4
          903

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy