Welcome to the Q&A Forum

Hi Dmitri and Juan - perhaps this way of explaining it will help:

A good analogy might be how rankings work for countries in various categories. For example, if Japan is ranked as having the world's best healthcare in 2015, and they improve the quality of their healthcare in 2016, are they guaranteed to still be #1?

Not necessarily.

Maybe the #2 ranked country improved even more and now Japan has fallen from #1 to #2 despite actually improving on their healthcare quality. Maybe countries 2-10 all improved dramatically and Japan's now fallen to #11 even though they technically got better, not worse.

PA and DA work in a similar fashion. Since they're scaled on a 100-point system, after each update, the recalculations mean that PA/DA for a given page/site could go down even if that page/site has improved their link quantity and quality. Such is the nature of a relative, scaled system. This is why I encourage folks strongly to watch not just PA/DA for their own pages/sites, but for a variety of competitors and sites in similar niches to see whether you're losing or gaining ground broadly in your field.

The score system has to be relative (we can't use absolutes or we wouldn't be able to have good correlations against Google - we'd just have another system that counts links or counts linking domains or the like). If PA/DA aren't working well for you as metrics, I'd encourage you to use something else - link counts or counts of linking domains/IPs or the like. The purpose of the DA/PA metrics is to track against Google, and over time, you might see lots of fluctuation up or down that doesn't necessarily mean you're doing better/worse. That's why the comparison process and understanding what the metrics do and why they're different than raw counts is important.

Hope that's helpful!

UPDATE 12/01/15: This issue is now resolved and Just Discovered Link data is back.

Sadly, I've got bad news. Due to an error we made (literally a typo on a curl command), Just Discovered Links in Open Site Explorer will be unavailable for 2-3 days. The data was actually lost, but we're re-indexing all those links now and they should be back in working order by Monday.

NOTE: this only affects link counts and links listed in the "Just Discovered" sections of OSE. No links in the main index nor any metrics (PA/DA/etc) are affected.

Our sincere apologies - we're going to build mechanisms to prevent this for the future.

If you don't mind the loss of ranking signals between the subdomains/domains, and are simply seeking to dominate the search results through owning multiple positions, separate domains are the best way to go. Subdomains can work for this, too, but are less consistently treated as separate sites, and Google may change that in the future.

Thanks for the feedback Joseph - I appreciate your transparency and can totally empathize with the frustration.

I think the key here, unfortunately, is in understanding and effectively explaining how the metrics of DA and PA operate and why they're not like standard counts that always go up as things get better. Clearly, we need to do a better job of that.

A good metaphor might be how rankings work for countries in various categories. For example, if Japan is ranked as having the world's best healthcare in 2015, and they improve the quality of their healthcare in 2016, are they guaranteed to still be #1?

Not necessarily.

Maybe the #2 ranked country improved even more and now Japan has fallen from #1 to #2 despite actually improving on their healthcare quality. Maybe countries 2-10 all improved dramatically and Japan's now fallen to #11 even though they technically got better, not worse.

PA and DA work in a similar fashion. Since they're scaled on a 100-point system, after each update, the recalculations mean that PA/DA for a given page/site could go down even if that page/site has improved their link quantity and quality. Such is the nature of a relative, scaled system. This is why I encourage folks strongly to watch not just PA/DA for their own pages/sites, but for a variety of competitors and sites in similar niches to see whether you're losing or gaining ground broadly in your field.

Hope that's helpful and wish you all the best.

Hi Joe - yes, it's most likely the fluctuations are because you're in that lower range. Remember that Moz can't see links you've disavowed in Google Webmaster Tools/Search Console, so we wouldn't be lowering DA/PA based on those (although, it's possible that, over time, as Google stopped counting links like that, our algorithm for PA/DA would "learn" and evolve, but that would take a while).

Hi Donna - yes, that fluctuation should be much larger on average in the tail of the web (sites with DA 0-40) than in the middle or head. This makes sense because with a relative metric, all of the factors I describe above are going to be magnified in the tail, particularly because Google's rankings change so much there and because just a few links can have such a huge impact. For metrics-savvy clients, they should be best poised to understand that since DA/PA are exponential, a few links here or there and a few valuation shifts on those links can have big swings in the 10-40 point ranges of DA/PA, whereas in the 50/60+ ranges, small shifts in link discovery or in link valuation (from us or Google) won't have as much change.

As far as a ceiling - no, we don't have a recommendation there. The idea is that as DA/PA fluctuate, especially as they get more accurate in predicting rankings (correlations & coverage), the fluctuations are generally happening because Google's changing and we're getting closer to tracking how and where (with exceptions I noted above around issues with our crawl/indexing). My biggest recommendation is to keep track of similar-sized competitors (and larger/smaller ones) so you've got a set of benchmarks for comparison.

Thanks for the note and apologies for the frustrations this causes.

Always happy to help, and especially to provide transparency. Thanks for the kind response

Hi Deacyde - that shouldn't be the message you're taking away from this post at all! As I noted above, you could improve the SEO, improve your link profile, and still see a reduction in Domain Authority as as a score due to how DA is done (on a relative scale, not an absolute scale). We could find more and better links to your site, and you'd still see a lower DA.

If a number of smaller sites in your field have all seen lower DA score in this index, that's indicative that it's nothing you've done, but rather, an indication that DA has been shifting across the board. If you're seeing rankings stay high and organic search traffic stay high, then there's nothing to worry about, and DA should still work just as well as a relative metric across sites (actually, slightly better given the improved correlations).

Howdy folks,

Every time we do an index update here at Moz, we get a tremendous number of questions about Domain Authority (DA) and Page Authority (PA) scores fluctuating. Typically, each index (which release approximately monthly), many billions of sites will see their scores go up, while others will go down. If your score has gone up or down, there are many potential influencing factors:

1. You've earned relatively more or less links over the course of the last 30-90 days.
   Remember that, because Mozscape indices take 3-4 weeks to process, the data collected in an index is between ~21-90 days old. Even on the day of release, the newest link data you'll see was crawled ~21 days ago, and can go as far back as 90 days (the oldest crawlsets we include in processing). If you've done very recent link growth (or shrinkage) that won't be seen by our index until we've crawled and processed the next index.
2. You've earned more links, but the highest authority sites have grown their link profile even more
   Since Domain and Page Authority are on a 100-page scale, the very top of that represents the most link-rich sites and pages, and nearly every index, it's harder and harder to get these high scores and sites, on average, that aren't growing their link profiles substantively will see PA/DA drops. This is because of the scaling process - if Facebook.com (currently with a DA of 100) grows its link profile massively, that becomes the new DA 100, and it will be harder for other sites that aren't growing quality links as fast to get from 99 to 100 or even from 89 to 90. This is true across the scale of DA/PA, and makes it critical to measure a site's DA and a page's PA against the competition, not just trended against itself. You could earn loads of great links, and still see a DA drop due to these scaling types of features. Always compare against similar sites and pages to get the best sense of relative performance, since DA/PA are relative, not absolute scores.
3. The links you've earned are from places that we haven't seen correlate well with higher Google rankings
   PA/DA are created using a machine-learning algorithm whose training set is search results in Google. Over time, as Google gets pickier about which types of links it counts, and as Mozscape picks up on those changes, PA/DA scores will change to reflect it. Thus, lots of low quality links or links from domains that don't seem to influence Google's rankings are likely to not have a positive effect on PA/DA. On the flip side, you could do no link growth whatsoever and see rising PA/DA scores if the links from the sites/pages you already have appear to be growing in importance in influencing Google's rankings.
4. We've done a better or worse job crawling sites/pages that have links to you (or don't)
   Moz is constantly working to improve the shape of our index - choosing which pages to crawl and which to ignore. Our goal is to build the most "Google-shaped" index we can, representative of what Google keeps in their main index and counts as valuable/important links that influence rankings. We make tweaks aimed at this goal each index cycle, but not always perfectly (you can see that in 2015, we crawled a ton more domains, but found that many of those were, in fact, low quality and not valuable, thus we stopped). Moz's crawlers can crawl the web extremely fast and efficiently, but our processing time prevents us from building as large an index as we'd like and as large as our competitors (you will see more links represented in both Ahrefs and Majestic, two competitors to Mozscape that I recommend). Moz calculates valuable metrics that these others do not (like PA/DA, MozRank, MozTrust, Spam Score, etc), but these metrics require hundreds of hours of processing and that time scales linearly with the size of the index, which means we have to stay smaller in order to calculate them. Long term, we are building a new indexing system that can process in real time and scale much larger, but this is a massive undertaking and is still a long time away. In the meantime, as our crawl shape changes to imitate Google, we may miss links that point to a site or page, and/or overindex a section of the web that points to sites/pages, causing fluctuations in link metrics. If you'd like to insure that a URL will be crawled, you can visit that page with the Mozbar or search for it in OSE, and during the next index cycle (or, possibly 2 index cycles depending on where we are in the process), we'll crawl that page and include it. We've found this does not bias our index since these requests represent tiny fractions of a percent of the overall index (<0.1% in total).

My strongest suggestion if you ever have the concern/question "Why did my PA/DA drop?!" is to always compare against a set of competing sites/pages. If most of your competitors fell as well, it's more likely related to relative scaling or crawl biasing issues, not to anything you've done. Remember that DA/PA are relative metrics, not absolute! That means you can be improving links and rankings and STILL see a falling DA score, but, due to how DA is scaled, the score in aggregate may be better predictive of Google's rankings.

You can also pay attention to our coverage of Google metrics, which we report with each index, and to our correlations with rankings metrics. If these fall, it means Mozscape has gotten less Google-shaped and less representative of what influences rankings. If they rise, it means Mozscape has gotten better. Obviously, our goal is to consistently improve, but we can't be sure that every variation we attempt will have universally positive impacts until we measure them.

Thanks for reading through, and if you have any questions, please leave them for us below. I'll do my best to follow up quickly.

Hi Joseph - yes, I can answer that. We took ~14 days to process this latest index, which is very good news. However, we are having some trouble with the uploading process again - our technical operations team is working with the big data team to try and uncover the source of these problems. If we can get it fixed and working (in the past, the upload step took ~12 hours, now it's taking us 3-4 days), we should have much more regular index releases.

Right now, we are feeling confident about Nov. 17th, and once we complete the upload we'll have a good picture about data quality and whether we might be able to release early (which we think is quite possible IF quality looks good and these upload issues get sorted).

I hope we might actually have that 11/17 index out a little bit early. We've made a lot of fixes and optimizations, and, fingers crossed, it looks (so far) like it's making a difference in terms of speed to index processing completion.

That doesn't surprise me - Majestic has a larger index than Moz (theirs is actually the largest among active 3rd party indices, then Ahrefs, then us).

https://moz.com/blog/big-data-big-problems-link-indexes-compared this is a pretty good resource comparing the strengths and weaknesses of the various indices, and https://builtvisible.com/comparing-link-data-tools/ is also a good, third-party review of the three. There are strengths and weaknesses to each, but if raw link coverage is your goal, I recommend Majestic.

There are a variety of reasons that include:

1. This index is somewhat smaller in total links crawled and URLs included
2. We may have biased the crawlers towards sites/pages that are less likely to feature links from your site (this is particularly possible if the linking sites were on Chinese, Palau, or several other TLD extensions that we had previously over-indexed)
3. The links we previously crawled may have been on relatively low MozRank pages that this index didn't crawl b/c we found fewer links to them (and thus lower MozRank - we tend to crawl in roughly descending MozRank order across the web).

As noted, the next index should see better coverage, fresher data, and better metrics, too. Please let us know if the problem persists and maybe we can compare your WM Tools links vs. our index to see what could be happening. Thanks and apologies.

We are both in the same boat there. I, too, desperately need this next index to get us on track and provide excellent value. If not, I think we're going to lose a lot of customers, and I'm not sure people will trust us for a long time on link data.

You have my deep and sincere apologies for the frustration and professional challenge Moz has caused. We have an obligation to do better, and I damn sure hope the team is up to delivering on that obligation.

Hi Joseph - you'll get no arguments from me on any of these fronts. I think if you've been using Moz exclusively or primarily for the link data component, you should request a refund by emailing help@moz.com (they'll be happy to provide one). Totally concur that our service the past 60 days on the link data front has not been acceptable.

Hi Donna - you are most certainly not alone in your frustration. I would call my own feelings bordering on  desperation. I'm frustrated, angry, nervous, guilty, and overwhelmed with a sense of powerlessness. It seems that every time we think we've identified a problem at the root of our Mozscape issues, things just get worse and new problems we never imagined arise.

On the padding issue, I have good news and depressing news. The good news is that we pad every estimate by nearly 2X. In a normal, problem-free index cycle, we can get it done in 12-14 days.... And yet, we never estimate less than 30-31 days for an index release. In the early part of this year, you might recall that we had a number of indices released back to back in that 2-3 week window. Then things took a turn for the worse and we've been struggling ever since.

I want to be honest - my belief is that we are going to get better, but the evidence of the last 6 months is against me. I want to believe my team and I know they are trying hard and doing everything they can to get this fixed. However, I think it's wise to have skepticism given the trajectory of the recent past.

Hope that's helpful and thank you for the comment.

Howdy gang,

As you might have seen, we've finally been able to update the Mozscape index after many challenging technical problems in the last 40 days. However, this index has some unique qualities (most of them not ideal) that I should describe.

First, this index still contains data crawled up to 100 days ago. We try to make sure that what we've crawled recently is stuff that we believe has been updated/changed, but there may be sites and pages that have changed significantly in that period that we didn't update (due to issues I've described here previously with our crawlers & schedulers).

Second, many PA/DA and other metric scores will look very similar to the last index because we lost and had problems with some metrics in processing (and believe that much of what we calculated may have been erroneous). We're using metrics from the prior index (which had good correlations with Google, etc) until we can feel confident that the new ones we're calculating are correct. That should be finished by the next index, which, also, should be out much faster than this one (more on that below). Long story short on this one - if your link counts went up and you're seeing much better/new links pointing to you, but DA/PA remain unchanged, don't panic - that's due to problems on our end with calculations and will be remedied in the next index.

Third - the good news is that we've found and fixed a vast array of issues (many of them hiding behind false problems we thought we had), and we now believe we'll be able to ship the next index with greater quality, greater speed, and better coverage. One thing we're now doing is taking every URL we've ever seen in Google's SERPs (via all our rank tracking, SERPscape, the corpus for the upcoming KW Explorer product, etc) and prioritizing them in Mozscape's crawl, so we expect to be matching what Google sees a bit more closely in future indices.

My apologies for the delay in getting this post up - I was on a plane to London for Searchlove - should have got it up before I left.

Hi Mick - I'd say that if you're not finding value from Google+, I wouldn't worry about it. Google is investing less in it and unless you're in a community/niche where G+ is heavily used (travel, photography, online marketing are a few), probably best to put your effort elsewhere. The only real SEO benefit to G+ left is the personalization of SERPs, but that will only work if you have a large group following you who can be biased in their SERPs by their connection with you. Don't think it's geographically specific, but it's certainly niche-specific.

Best of luck!

Sometimes yes. Sometimes, we don't know until we reach the last stages of processing whether it's going to finish or take longer. We're trying to get better at benchmarking along the way, too, and I'll talk to the team about what we can do to improve our metrics as an index run is compiling.

It didn't break, but it is taking longer to process than we hoped. Very frustrating, but we have a plan that, starting in a few more weeks, should get us to much more consistent index releases (and better quality ones, too).