Why Does OSE (Open Site Explorer) have such little backlink data on russian sites in the google.ru index?

[Turkey]

OK this seems v strange, but google.ru are indexing far more BLs in their SERPS  for a widget than OSE reports. Very little data is found in OSE for russian based sites.

Is this the marketing intention?

(I could send raw data if needed!)

What is filtering this vast google.ru data list out? Is OSE only catered for US/UK?

[Cyrus-Shepard]

Hi Turkey,

Good observation. It's certainly not an intentional bias. OSE is designed as a world-wide link index. That said, there are a couple of forces at play here.

First, because of the vast amount of links on the web,the OSE index is designed to find the most significant links. These are the links most likely to influence rankings. With this in mind, OSE usually only contains 40-60% of the links recorded by Google Webmaster Tools. This is true in all regions, including Europe and the United States.

The good news is that the majority of the time, the missing links are very often the same ones that pass little or no value.

It is possible certain "pockets" of the web get less bandwidth from Linkscape crawlers than other areas, due to natural selection. Linkscape starts with a "seed" of trusted sites, which include the top sites from the last index, and then crawls "out" from there. This method means sites well linked to from the "seed" sites and other top-ranked sites have a higher likelyhood of being crawled each index. If Linkscape doesn't have good metrics for a particular site, it is most likely distanced from the seeds.

Every so often the seed sites are adjusted, and there is a natural selection, in order to keep the index fresh and relative. It's one of the best indexes of its kind in the world, but of course there is always room for improvement.

Hope this explination helps. Thanks again for the feedback.

[Turkey]

Thanks for your response (last May - a while back I know)

I re-read your answer and it got me thinking - does OSE only find trusted seed set sites in the eyes of Google? Or does it consider beginning its 'trust' quest journey by what other engines consider,   or humans even??

If google turned on a 'cynical' algorithm - surely it would consider most of the newspaper and media links as 'gamed' - turn these off - reverse the polarities and go to find the far remote honest and unbias independent citations from deeper parts of the web. How does one judge what a seed site should be in each country? Is this a manual choice?

[Cyrus-Shepard]

I don't know the exact composition of seed sites that SEOmoz uses, but it's believed to be similar, at least in theory to a process that search engines use.

This would include seed sites such as universities and government agencies and other highly respected sites, for example the Red Cross would be considered a high trust site that would make a good seed.

The set is updated frequently as sites can be gamed. But I wouldn't go as far to say that all newspapers and media sites go in the "bad" category. Some do, some don't.

It's funny that you mention a "cynical" algorithm. It's believe Google does use something similar to a "SpamRank" algorythm.