Good crawlers are always useful although real value depends on what aspects of a site you want to analyse and how much data there is to crawl. Xenu is great for quick and dirty link analysis, whereas Screaming Frog, (Microsoft's IIS) SEO Toolkit and DeepCrawl all include much more detail and analysis options. Also worth noting that Screaming Frog and SEO Toolkit run from your local machine, whereas DeepCrawl is cloud based and therefore more scalable. There might be other SEO crawlers out there too but these are the ones I use most.
Posts made by semetrical-197165
-
RE: Anybody using Screaming Frog?