Campaign Crawl

[JMFieldMarketing]

I have a site with 8036 pages in my sitemap index.  But the MozBot only Crawled 2169 pages.  It's been several months and each week it crawls roughly the same number of pages.  Any idea why I'm not getting fully crawled?

[Mike.Goracke]

Hi Jack,

Server errors. Redirect loops. Disallowed pages in robots.txt. etc.

Here are some suggestions from www.seomoz.org/help/crawl-diagnostics:

"Why didn’t you crawl all my pages? I only got a one page crawl. Looks like you missed a bunch!

If you suspect you didn’t get a full crawl, or Rogerbot missed some of your pages, there could be several reasons why this happens.

• We only crawl a maximum of 400 links per page. If several pages of your site all have the same 400 links on each page, we may not discover all the pages on your site. Try optimizing your navigation to reduce the number of links.
• Does your navigation rely on JavaScript? Can visitors navigate your site with JavaScript disabled? SEOmoz doesn’t crawl JavaScript, so make sure your links work in all browsing environments.
• Does your site consist of multiple subdomains? Crawls are restricted to the subdomain you set your campaign up on. This means that in general, we don't crawl multiple subdomains. You can solve this by specifying a “Root Domain” crawl in the setup process. (This requires starting a new campaign.)"

If you are certain you don't have any of the above problems, then I would suggest contacting help@seomoz.org

Good luck.

Mike

[josh-riley]

The best and most efficient solution with any question regarding Moz crawls or tools is to go directly to them with questions (sometimes there's enough unique behind the scenes stuff going on that forum people don't have access to). Their team is great and has helped me a handful of times, quickly and politely.