The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Technical SEO Issues
    4. What crawler do you recommend for finding orphaned pages on a website?

    What crawler do you recommend for finding orphaned pages on a website?

    Technical SEO Issues
    3 3 350
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • DigitalMarketingSEO
      DigitalMarketingSEO last edited by

      Is there a crawler that you guys recommend for finding all pages, including orphaned pages on a website? A data export is not feasible. I saw a question from back in 2013 and was wondering if anything has changed since then in regards to crawling orphaned pages. Do most enterprise systems already have this built into their crawler? Or is it best to get a crawler like Xenu or Screaming Frog or Deepcrawl?

      1 Reply Last reply Reply Quote 0
      • PatrickDelehanty
        PatrickDelehanty last edited by

        Hi there

        I really like ScreamingFrog but I also really like Search Console and Moz Pro. The reason being, I like having different sets of data because they are all different. I also like seeing if pages are being linked to randomly from other sources other than my own website which Search Console does a great job (and so does Majestic or Ahrefs). Different sources find different things so it's nice to get other opinions on what you might have out there floating around.

        Just my two cents! Hope this helps!
        Patrick

        1 Reply Last reply Reply Quote 4
        • BlueCorona
          BlueCorona last edited by

          Hi there!

          i agree with Patrick. I was going to recommend using Screaming Frog or Google Search Console! Let me know if you try these, don't like them, and need another recommendation.

          1 Reply Last reply Reply Quote 2
          • 1 / 1
          • First post
            Last post
          • My WP website got attack by malware & now my website site:www.example.ca shows about 43000 indexed page in google.
            0
            1
            82

          • Page Rank Being Split Between www.website.com and http://website.com
            DmitriiK
            DmitriiK
            0
            2
            25

          • Redesigned and Migrated Website - Lost Almost All Organic Traffic - Mobile Pages Indexing over Normal Pages
            DirkC
            DirkC
            0
            4
            245

          • Website SEO Product Pages - Condense Product Pages
            Ed-iOVA
            Ed-iOVA
            0
            4
            144

          • My beta site (beta.website.com) has been inadvertently indexed. Its cached pages are taking traffic away from our real website (website.com). Should I just "NO INDEX" the entire beta site and if so, what's the best way to do this? Please advise.
            Vuly
            Vuly
            0
            5
            1.6k

          • What is the best way to find missing alt tags on my site (site wide - not page by page)?
            franchisesolutions
            franchisesolutions
            1
            4
            9.7k

          • Website On Page Analysis
            KeriMorgret
            KeriMorgret
            0
            4
            323

          • Our Development team is planning to make our website nearly 100% AJAX and JavaScript. My concern is crawlability or lack thereof. Their contention is that Google can read the pages using the new #! URL string. What do you recommend?
            john4math
            john4math
            0
            2
            651

          Get started with Moz Pro!

          Unlock the power of advanced SEO tools and data-driven insights.

          Start my free trial
          Products
          • Moz Pro
          • Moz Local
          • Moz API
          • Moz Data
          • STAT
          • Product Updates
          Moz Solutions
          • SMB Solutions
          • Agency Solutions
          • Enterprise Solutions
          • Digital Marketers
          Free SEO Tools
          • Domain Authority Checker
          • Link Explorer
          • Keyword Explorer
          • Competitive Research
          • Brand Authority Checker
          • Local Citation Checker
          • MozBar Extension
          • MozCast
          Resources
          • Blog
          • SEO Learning Center
          • Help Hub
          • Beginner's Guide to SEO
          • How-to Guides
          • Moz Academy
          • API Docs
          About Moz
          • About
          • Team
          • Careers
          • Contact
          Why Moz
          • Case Studies
          • Testimonials
          Get Involved
          • Become an Affiliate
          • MozCon
          • Webinars
          • Practical Marketer Series
          • MozPod
          Connect with us

          Contact the Help team

          Join our newsletter
          Moz logo
          © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
          • Accessibility
          • Terms of Use
          • Privacy