Difference between SEOMOZ and Webmaster Tools information

[ciznerguy]

Hello,

There is an issue that confuses me and I thought perhaps you will be able to help me shed some light on it.

I have a website which shows 2,549 crawled pages on SEOMOZ and 24,542 pages on webmaster tools!

Obviously there is some technical issue with the site, but my question is: why the vast difference between what the SEOMOZ crawl report and webmaster tools report show?

Thanks!

Guy Cizner

[Theo-NL]

Either those 'issues' might be the cause. For example incorrect canonicalization that is picked up differently by Google and the SEOmoz bot Roger. Another option could be that Google tries really hard to index each and every page of the web, while Roger has a slightly more restrictive way of crawling the web by only crawling pages above a certain level of authority / only a certain amount of clicks from the homepage etc.

[TomRayner]

The Mozscape index, as brilliant as it is, can in no way compete with the size of the index that Google can handle.

As a result, your WMT report should always have a bigger amount of pages, links etc crawled.  It's just bigger.

[Jeepster]

This may also shed some light:

Oct 9, 2012 Keri Morgret On-site Community Manager at SEOmoz:

Another reason is that we just don't have the same size server farm that Google and Bing have. We could crawl all of Twitter and get nothing else crawled, or we could crawl some of Twitter, and some of the rest of the web. We aren't able to crawl all of the web, and we release a new index about once a month, so that's why you don't see all of your links or see them right away.

However, what we do offer that is different from Google and Bing is that we show you links for sites that are not your own, we add metrics about the trust and authority of the page, etc.

 

[ciznerguy]

hello

thanks for all the replies.

the pages crawled are part of an SEO I am running.

How the crawl is done when a campaign is defined?

I assume all the site is being crawled.

thanks

[KeriMorgret]

Thanks for stepping in everyone, though it looks like we were trying to answer the wrong question. This one is with Roger's crawl of the OP's own site, rather than links indexed in OSE.

Guy, do you have a feel for how many pages SHOULD be in the index? If you only have a couple of thousand pages, then it could be that Google is crawling and indexing some parameters. If you've got 20k+ pages in the index, then Roger isn't finding some things.

Also..are you looking at perhaps just the www.domain subdomain in SEOmoz and is GWT looking at the entire site? If you had a compact www.domain site, but then had forum.domain and wiki.domain, and GWT was reporting pages for all of the subdomains on domain.com, that would explain things too.