Where is the keyword difficulty tool data sourced from?

[ZakGottlieb71]

I also use Market Samurai, and I've noticed what seem to be big discrepancies with the keyword data presented by this (data comes from Majestic SEO) and the Keyword Difficulty Tool.

To take just one example, I analyze the term "how to remove tea stains" In the Keyword Difficulty Tool, this returns the following:

Root Domain Linking Root Domains: 2,233

Page Linking Root Domains: 4

When I use Market Samurai, however, the data returned is:

RDD (Domains linking to this domain): 19,911

RDP (Domains linking to this page): 19

I thought that these two metrics were the same for both tools, but I've written them out in case someone sees a difference. As I say, Market Samurai data is sourced from Majestic SEO - a reputable SEO company - but I have no idea where the Keyword Difficulty Tool data is from, nor why these differences are so pronounced? Are they indeed the same metrics in both cases, or am I missing something?

Any insight would be much appreciated.

[DanDeceuster]

Any linking data presented by SEOMoz will be from their own Linkscape Index. This data is also presented at opensiteexplorer.org. If you see any SEOMoz tool showing links, linking domains or anything related to link metrics, that data comes from their proprietary Linkscape Index.

[Cyrus-Shepard]

Daniel gave the correct answer, all SEOmoz link metrics come from the Linkscape index. This is the same data found in Open Site Explorer, the API, the MozBar, SEOmoz PRO platform and all other SEOmoz tools.

All indexes (including Google, Yahoo, and Linkscape) count links differently. The vast majority of links on the web are "junk" that quickly disappeer and count very little or none at all for SEO purposes, so a big part of the Linkscape Index is an iterative process of "cleaning" and verifying the links before the Index is published every few weeks.

This difficulty was highlighted by our friends at Majestic when they wrote a guest post for the SEOmoz blog on Why Counting Links is not So Easy.

Regarding the Keyword Difficulty Tool, link metrics are only one part of a larger formula for calculating difficulty. Because all queries use the same Linkscape data to calculate difficulty, the score is relative so changes in link counts generally don't become a factor.

[ZakGottlieb71]

Hi Cyrus,

Thanks for this. I understand that counting links accurately is hard work, but if I am take this correctly, the above metrics are identical for SEOMoz and Majestic, which is a real head-scratcher, since it would suggest an enormous discrepancy in the data being presented, for almost every keyword link I have tested this on. To take just one other (basically random) example: "how to remove rust stains"

8th down in the first SERP is:

http://www.apartmenttherapy.com/la/how-to-remove-rust-stains-from-stainless-steel-home-hacks-108429

Which returns either THIS data (Linkscape):

Root Linking Root Domains  (total # of unique domains with backlinks pointing to this specific domain?): 15,471

Page Linking Root Domains (total # of unique domains with backlinks pointing to this page?): 91

Or this data (Majestic):

RDD (same definition as RLRD, above): 71, 152

RDP (same definition as for PLRD, above): 6

I encourage you to register for Majestic's services and try this out for yourself if this is news to you. In default of further knowledge of the specific methodologies used by these two different link-measuring tools,  if one were otherwise to consider them both equal, I don't see how any use could be made of this data, since it is so vastly different in each case? I still can't help feeling that I'm missing something...

Thank you again for your input on this - it's much appreciated.

Best,

Zak

[Cyrus-Shepard]

Yes, there is a discrepancy between all indexes on link counts, even though everyone might agree on the definition. But this doesn't mean one is right and one is wrong.

Counting links is like counting cats in Schrödinger's box. It changes depending on how you look at it... even though we all agree on the definition of a cat.

Using the same definitions as you described above, if you were to look at Google's index you would arrive at a number higher than this, and using Bing's index perhaps a number somewhere in the middle.

For example, consider that 50% of the web disappears every year, which means around 4% of all the links on the Internet disappear every month. But what if your crawlers only visit those links every 60 days? (this is an extreme example)

Or the fact that 80% of the web is spam that shouldn't be crawled. Depending on your definition of spam/trust, different crawlers will crawl different websites and arrive at different link counts, even though they are using the exact same definition of a link.

The important thing to remember is this - raw link counts are hardly ever useful. What is important is finding the links that matter. That's why using metrics such as Domain and Page Authority become increasingly important. 10 links can be more important that 10 million links, if they are the right ones.

That said, metrics like # of linking root domains do play a significant role in corollary ranking factors.

[ZakGottlieb71]

Ok, thanks. I get the overall point you're making, and will indeed focus more on link quality... I guess my gripe is that if the # of linking root domains (for example) are to be counted at ALL - even as a corollary ranking factor - then it would be really good to be sure of the accuracy of the data, which it's hard to be when Majestic, for example, presents such different numbers in each case.