The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Getting Started
    4. I’d like to set up a Moz campaign that crawls just the primary website, not subdomains

    I’d like to set up a Moz campaign that crawls just the primary website, not subdomains

    Getting Started
    2 2 32
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • blueprintatl
      blueprintatl last edited by

      Is this something you could help with? (it either bombs or crawls everything, so I assume I’m missing something in the campaign settings, or it’s just not possible.

      1 Reply Last reply Reply Quote 0
      • dave.kudera
        dave.kudera last edited by

        Hey, thanks for reaching out to us!

        You can block our crawler from crawling specific subdomains of your site by excluding our user-agent from the subdomains robots.txt file.  Our user agent is "rogerbot" and the directive would look something like this:

        User-agent: rogerbot
        Disallow: /

        You will want to make sure that this is associated with the robots.txt file specific to the subdomain you want to block.

        You can read more about Rogerbot in our guide.

        Hope that helps, let me know if you have further questions.

        1 Reply Last reply Reply Quote 0
        • 1 / 1
        • First post
          Last post
        • Moz only crawling one page of a campaign, please help
          dave.kudera
          dave.kudera
          0
          2
          122

        • My question is, when you translate your website to another language, does moz crawl both or do i have to add another campaign to moz so that they can crawl it seperately?
          RyanPurkey
          RyanPurkey
          0
          4
          130

        • My website does not allow all crawler to crawl, Now my question is that whether i need to give permission to moz crawler if yes then whaat is moz bot name?
          MonicaOConnor
          MonicaOConnor
          0
          3
          794

        • I am managing an existing ecommerce website and just subscribed to the MOZ tools - what is the best rout to learning how bets leverage all the tools to optimize my site?
          LesleyPaone
          LesleyPaone
          0
          4
          135

        • Can I use wildcards "*" when setting up a new Moz campaign?
          Cyrus-Shepard
          Cyrus-Shepard
          0
          4
          168

        • Moz Analytics set up: should I use www or not when I do website set up?
          HST-Marketing
          HST-Marketing
          0
          8
          2.4k

        • Can MOZ crawl our website twice in a week?
          chandman
          chandman
          0
          4
          146

        • Where to find answers to really dumb questions about setting up Moz campaigns...
          RobertFisher
          RobertFisher
          0
          2
          162

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy