Does it really take 3-7 days to crawl a site?

[rldonnell]

This is what you say on your website:

Please note: depending on the size and speed of your site, it may take between 3 and 7 days to complete your crawl.

Are you guys kidding?  This is unacceptable for almost all of my deadlines, and I suspect 99% of the SEO world...  you can fly to the moon faster.

[MarloSchneider]

The time it takes is not based off of your site alone, there are so may sites and links being crawled which all take processing power and bandwidth.

[randfish]

I hear you - it's a frustration, but in a system that needs to crawl 30K+ sites each week without hitting anyone's server too hard, being a bother to a SysAdmin and keeping up with a massively complex, ever-changing queue, it's the reality.

That said, we do have a custom crawl tool you can fire anytime and usually get data back within just a few hours! It's here - http://www.seomoz.org/labs/cc

The intent/value behind the crawl inside the campaign is much less about a specific, one-time crawl, and more about having data every week, with historical information showing progress, updates and warnings (in case something goes wrong). There's lots of good free tools as well for single-purpose crawls, e.g. http://www.seomoz.org/blog/xenu-link-sleuth-more-than-just-a-broken-links-finder

Also, just to be totally clear - the system running the crawls for the web app in you campaigns is different than the Linkscape web index (which only updates every 3-4 weeks). Eventually, the two might merge, but we didn't want to bias any crawling inside Linkscape when we launched the web app last September.