The Moz Q&A Forum

    • Forum
    • Questions
    • My Q&A
    • Users
    • Ask the Community

    Welcome to the Q&A Forum

    Browse the forum for helpful insights and fresh discussions about all things SEO.

    1. SEO and Digital Marketing Q&A Forum
    2. Categories
    3. Online Marketing Tools
    4. URL crawler?

    URL crawler?

    Online Marketing Tools
    5 3 886
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as question
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • Horizon
      Horizon last edited by

      All,

      I am looking for a software that can crawl a specified URL and then report all incoming links to that URL.

      For example. I would like to find out all incoming links to http://www.mysite.com/article1. Preferably internal and external links. So I might expect to a list like this:

      http://www.mysite.com/

      http://www.mysite.com/article2

      http://www.mysite.com/article3

      http://www.mysite.com/article11

      http://www.anothersite.com/article199

      Does such a thing exist? I have Xenu Link Sleuth, but this crawls all links on my site and just reports on broken links, which isn't the most helpful tool for the purpose I need it for...

      Thanks

      Matt

      1 Reply Last reply Reply Quote 0
      • gfiorelli1
        gfiorelli1 last edited by

        Not totally sure if my answer is correct, but I do believe that IIS SEO Toolkit allows you to do that kind of verification.

        Edit: stupid me > OSE

        1 Reply Last reply Reply Quote 0
        • Horizon
          Horizon last edited by

          Thanks Gianluca. IIS access isn't available to me at this particular moment, unfortunately.

          Are you (or anyone else) aware of any third party software?

          Thanks

          1 Reply Last reply Reply Quote 0
          • Marcus_Miller
            Marcus_Miller last edited by

            Maybe I am missing something here but can you not use opensiteexplorer.org?

            So, you want all links to say this page from internal and external sources:
            http://www.seomoz.org/blog/how-much-does-google-make

            A standard search (default settings in open site explorer) gives you just that:
            http://www.opensiteexplorer.org/links?site=www.seomoz.org%2Fblog%2Fhow-much-does-google-make

            You can then just dump all this info out to a spreadsheet and manipulate it as needed.

            Hope that helps.
            Marcus

            1 Reply Last reply Reply Quote 1
            • Horizon
              Horizon last edited by

              Thanks for all of the responses. I wasn't aware of opensiteexplorer.org and it does seem pretty handy. Not sure I 100% trust it (a few common sense checks displays some, but not all links), but it's a good starting point.

              Cheers

              M

              1 Reply Last reply Reply Quote 0
              • 1 / 1
              • First post
                Last post
              • URLS in search results.
                jasongmcmahon
                jasongmcmahon
                0
                2
                61

              • What is the name of BrightEdge's crawler?
                GastonRiera
                GastonRiera
                0
                2
                351

              • Pointing multiple urls to the same site
                PatrickDelehanty
                PatrickDelehanty
                0
                2
                160

              • Spell Check Crawler
                Caro-O
                Caro-O
                0
                5
                298

              • Url grabber?
                Travis_Bailey
                Travis_Bailey
                0
                5
                1.4k

              • Advanced Regex Question for URL Matching
                xerox432
                xerox432
                0
                4
                178

              • TLD URL List Tool?
                mihaiaperghis
                mihaiaperghis
                0
                5
                233

              • Unique URLs in Google Shopping Feed
                Everett
                Everett
                0
                3
                1.4k

              Get started with Moz Pro!

              Unlock the power of advanced SEO tools and data-driven insights.

              Start my free trial
              Products
              • Moz Pro
              • Moz Local
              • Moz API
              • Moz Data
              • STAT
              • Product Updates
              Moz Solutions
              • SMB Solutions
              • Agency Solutions
              • Enterprise Solutions
              • Digital Marketers
              Free SEO Tools
              • Domain Authority Checker
              • Link Explorer
              • Keyword Explorer
              • Competitive Research
              • Brand Authority Checker
              • Local Citation Checker
              • MozBar Extension
              • MozCast
              Resources
              • Blog
              • SEO Learning Center
              • Help Hub
              • Beginner's Guide to SEO
              • How-to Guides
              • Moz Academy
              • API Docs
              About Moz
              • About
              • Team
              • Careers
              • Contact
              Why Moz
              • Case Studies
              • Testimonials
              Get Involved
              • Become an Affiliate
              • MozCon
              • Webinars
              • Practical Marketer Series
              • MozPod
              Connect with us

              Contact the Help team

              Join our newsletter
              Moz logo
              © 2021 - 2026 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
              • Accessibility
              • Terms of Use
              • Privacy