Opensite Explorer not picking up followed links

[SnapComms]

This post is deleted!

[samantha.chapman]

Hey there!

Sam from Moz's Help Team here - thanks for writing in with a great question!

This is definitely due to the way we collect this information for our link index and it is actually expected that we won’t find every page and link because we aren’t looking for them all! When we collect this data, we’re looking specifically for the most valuable links and, rather than crawling your entire site or every site, we collect this by starting our crawler on a few highest ranking sites and letting it perform a breadth first search to see what it finds.

For each page that we crawl, we first collect each of it’s links before following these and collecting the details of each page that these link to and so on. There’s a set limit of links that we’ll crawl per page and pages that we’ll crawl per site so it’s expected that we may not follow every link on a site this way.

Just a few points on how exactly how we compile our index:

• We grab the most recent index.
• We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains).
• We start crawling from the top down until we've crawled ~130 billion URLs

The idea here is that we're focusing on the highest-quality links we can find, coming from the most prominent pages of authoritative sites. So, while you may not see every link for a site within our index, we're aiming to report the most valuable ones available.

Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for many reasons - including the crawl-ability of sites, the number of inbound links to them, and the depth of pages in subdirectories.

Since Moz focuses on quality of links over quantity, we are always focused on the most relevant links to display to our users. It's possible that Moz's index will leave out some of the lower-quality (non-link juice providing) links out of our index because of this. So, that might explain why you may see some discrepancies with what other tools may be showing.

If you’re looking for a tool to show all of your backlinks, we might not be the best fit for you but we do hope to show you all of the most valuable backlinks.

I hope this helps - let me know if you have any further questions!