10/14 Mozscape Index Update Details

[NordicClick_Interactive]

Hi Rand,

We have noticed among many of our clients that there has been a drastic drop in inbound links since the update to the Mozscape index. Our DA/PA have stayed the same, but links have dropped approximatey 60-70% across the boards. We have looked into the issue by looking in Google Webmaster Tools and have found that we have quite a few more links than are showing up in Moz. Could you explain why this might be happening?

Thanks!

[IanWatson]

Hey There,

Sorry this wasn't called out, but the number of links in this index is quite a bit lower than our previous releases, due to some of the technical issues Rand calls out here and in the September post (https://moz.com/community/q/september-s-mozscape-update-broke-we-re-building-a-new-index). The next release will remedy the issues you are currently seeing, as we have identified and fixed the bugs that caused the issues. Sorry again for any confusion here.

[randfish]

There are a variety of reasons that include:

1. This index is somewhat smaller in total links crawled and URLs included
2. We may have biased the crawlers towards sites/pages that are less likely to feature links from your site (this is particularly possible if the linking sites were on Chinese, Palau, or several other TLD extensions that we had previously over-indexed)
3. The links we previously crawled may have been on relatively low MozRank pages that this index didn't crawl b/c we found fewer links to them (and thus lower MozRank - we tend to crawl in roughly descending MozRank order across the web).

As noted, the next index should see better coverage, fresher data, and better metrics, too. Please let us know if the problem persists and maybe we can compare your WM Tools links vs. our index to see what could be happening. Thanks and apologies.

[RandyMilanovic]

This post is deleted!

[RandyMilanovic]

Rand, I've sent info and s/s to Kevin (at Moz) in an attempt to find some commonality between GSC, MOZ and HubSpot. 3rd party tools are showing more links that OSE, and are more in line with GSC. For example, Majestic shows 5x the linking domains that OSE does (on the root, not the www). Kevin points at this thread and cites the present OSE data. I'm trying to figure out why redirected/GWT 'moved' domains don't add up links, or even if they are supposed to? I suspect he's powerless. Who/what can I trust?

[randfish]

That doesn't surprise me - Majestic has a larger index than Moz (theirs is actually the largest among active 3rd party indices, then Ahrefs, then us).

https://moz.com/blog/big-data-big-problems-link-indexes-compared this is a pretty good resource comparing the strengths and weaknesses of the various indices, and https://builtvisible.com/comparing-link-data-tools/ is also a good, third-party review of the three. There are strengths and weaknesses to each, but if raw link coverage is your goal, I recommend Majestic.

[JosephGourvenec]

Hi Rand,

I hope you're well and life is good.

I was wondering if you can shed some light on the upcoming OSE update scheduled for the 17th of November.

In an earlier post you said "The good news is that we pad every estimate by nearly 2X. In a normal, problem-free index cycle, we can get it done in 12-14 days."

This would indicate you have potentially / already run an update and reviewing the data to ensure it it correct and relevant to the masses, before making a general release.

Can you advise if the trial update has been run, a success or not, if not do you think you'll have the issue solved for the 17th of November 2015?

I'm very eager to report back to my clients with credible insight using the data you provide.

Cheers,
Joseph Gourvenec

[jennita]

Hi Joseph! I'm sure Rand will chime in as well, but I know our engineers are currently working on a write-up that explains the future of the index, plus some of the issues of the past. They're trying to get all the details in there, and hopefully we can get that published by early next week. What I know so far, is that they've fixed some issues and this index is looking much better. I'll let the engineers explain what that means though.

[randfish]

Hi Joseph - yes, I can answer that. We took ~14 days to process this latest index, which is very good news. However, we are having some trouble with the uploading process again - our technical operations team is working with the big data team to try and uncover the source of these problems. If we can get it fixed and working (in the past, the upload step took ~12 hours, now it's taking us 3-4 days), we should have much more regular index releases.

Right now, we are feeling confident about Nov. 17th, and once we complete the upload we'll have a good picture about data quality and whether we might be able to release early (which we think is quite possible IF quality looks good and these upload issues get sorted).

[JosephGourvenec]

Thank you Jennita and Rand for your quick responses. 

Great, lets keep our fingers crossed all goes well and I'm confident the Moz team can deliver it.

We all take a ride on the ebb and flow roller coaster from time to time, its what makes us learn more and overcome challenges.

Have a great day

Cheers,
Joseph