The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. Can I crawl a password protected domain with SEOmoz?

    Can I crawl a password protected domain with SEOmoz?

    Technical SEO Issues
    2 2 325
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • edlondon
      edlondon last edited by

      Hi everyone,

      Just wondered if anybody has been able to use the SEOmoz site crawler for password protected domains?   On Screaming Frog you are prompted for the username and password when you set the crawler running, however SEOmoz doesn't.  It seems you can only crawl sites that are live and publicly available - can anyone confirm if this is the case?

      Cheers,

      M

      1 Reply Last reply Reply Quote 0
      • TommyTan
        TommyTan last edited by

        Hi,

        I don't think it is possible to crawl any password protectec pages.  Password protected pages are usually blocked and bots are unable access those pages.

        And yes it is true that only sites that are publicly and live can be crawled.

        1 Reply Last reply Reply Quote 1
        • 1 / 1
        • First post
          Last post
        • Google Crawling Issues! How Can I Get Google to Crawl My Website Regularly?
          libero_net
          libero_net
          1
          8
          142

        • Old domain still being crawled despite 301s to new domain
          backcash
          backcash
          1
          10
          1.7k

        • How can the search engines can crawl my java script generated web pages
          randfish
          randfish
          0
          2
          1.3k

        • Can SEOMoz crawl a single page as oppose to an entire subfolder?
          CollinJarman
          CollinJarman
          0
          6
          318

        • Domain Crawl Question
          AppleCapitalGroup
          AppleCapitalGroup
          0
          3
          356

        • Can I do a redirect to a new domain name only a couple of weeks after having redirected to another domain?
          Francisco_Meza
          Francisco_Meza
          0
          4
          428

        • Will password protecting my test sub-domain help keep the SEs from indexing it?
          bloggidy
          bloggidy
          0
          6
          743

        • Seomoz api for domains working, for domains+directory not?
          KeriMorgret
          KeriMorgret
          0
          5
          742

        Get started with Moz Pro!

        Unlock the power of advanced SEO tools and data-driven insights.

        Start my free trial
        Products
        • Moz Pro
        • Moz Local
        • Moz API
        • Moz Data
        • STAT
        • Product Updates
        Moz Solutions
        • SMB Solutions
        • Agency Solutions
        • Enterprise Solutions
        • Digital Marketers
        Free SEO Tools
        • Domain Authority Checker
        • Link Explorer
        • Keyword Explorer
        • Competitive Research
        • Brand Authority Checker
        • Local Citation Checker
        • MozBar Extension
        • MozCast
        Resources
        • Blog
        • SEO Learning Center
        • Help Hub
        • Beginner's Guide to SEO
        • How-to Guides
        • Moz Academy
        • API Docs
        About Moz
        • About
        • Team
        • Careers
        • Contact
        Why Moz
        • Case Studies
        • Testimonials
        Get Involved
        • Become an Affiliate
        • MozCon
        • Webinars
        • Practical Marketer Series
        • MozPod
        Connect with us

        Contact the Help team

        Join our newsletter
        Moz logo
        © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
        • Accessibility
        • Terms of Use
        • Privacy