Welcome to the Q&A Forum

Hi there! This would be related to the fact that our index has a cache time associated with the links that we index. So, a link (or even site) could be removed from the web, but our index would not remove that link/site until the cache time has passed. Generally, this is in place to stabilize the index and avoid assumptions about missed links, but I can certainly see how it could cause some confusion.

Same here - score on Page Speed insights is pretty low as well - and also a timeout on webpagestest.org Dirk

Hi Well - given the troubles Moz has to build the regular index in the past months I doubt they will add this feature anywhere soon. Don't forget that DA & Trust is an internal metric of Moz - which tries to predict how well your site will rank in the SERP's. As all tools, it is just an estimate of the real thing. Dirk

Thanks Jo but there has already been an update since I started this thread. I wonder if UK sites get indexed less frequently in these updates. Thanks Neil

Hello! Thanks for writing in! There could be a couple of reasons, the first being that sadly our September index update broke I'd like to direct you towards Rand's Q&A post which discusses the latest regarding the Oct index update and our plans for the future :). The next update is scheduled for this week and you can always check our update schedule at any time, as well. If you're still not seeing your site in the index after the Oct index is released then it's likely that we've not been able to locate your pages through the links we follow to gather data for the index. We're not able to add sites by request. To give your site the best chance of being crawled and indexed it's worth double checking your site doesn't block web crawlers  (i.e., with "noindex" or Robots.txt). You'll also want to work on building links from sites with high Mozrank. I hope this helps!

I wonder whether Moz would count your social profiles in the website schema as links? Because then you wouldn't need to get rid of the official Facebook and Twitter buttons (if you don't want to). All you'd need to do is chuck this in your code: If Moz are just checking the source for the links, maybe that's all they'd need. And Schema is always good

I'm afraid that's the case if you're using Cloudfront Our system really ought to be able to handle that type of configuration, but it's proving to be quite an undertaking to make those changes from an engineering perspective. So, I'm not really able to say when we'll be able to accommodate SNI with our crawler. Sorry for the trouble!

Hi Emily Sorry to hear your site is not being crawled. The crawl is successful, however, your site is completely in javascript with no HTML links we can follow and unfortunately, our crawler only understands HTML. For more insight, please check out this post about our crawler here: https://moz.com/devblog/why-didnt-moz-crawl-my-site/ Hope this helps!

The Mozscape index is updated circa every 2 months.  The next update is on 9 October 2015, so check on the 10th and see what impact your new links have had.See the below link, https://moz.com/products/api/updates Additionally it often takes a couple of updates for all additional links to be included in the index. Sounds like you are on the right track, it just takes a bit of time for everyone else to catch up.. Hope that assists.

Thanks for this! I didn't think to check the server logs. I'll have them checked and make sure that it's not blocking Moz out from the crawl. We have thousands of URL's on our website and quite a strict security policy on the server - so I imagine Moz has probably been blocked out. Thanks, Liam

unless your talking about  URLs that are on your site it is describing inbound anchor text the answer would be number two Hope that helps, Tom

Hi there! While our index is rather large, it is by no means inclusive of every link on the web. We're continuously working to increase the scope of our index, which means that you could start to see more links included than before. Also, rather than a sheer volume change in our index, it could be related to the types of sites that we're looking at for a newer index update. Unfortunately, we don't have the ability to report when a link was discovered by the index (other than in 'Just-Discovered Links' as Dmitrii mentioned above). It's certainly something we'd like to be able to accommodate for the future, but those changes would be a ways down the road. With respect to a percentage increase, I don't believe it's displayed anywhere within the app, but you could find the value for that increase by dividing the new number of links (1130) by the original number of links (1130-875) and then multiplying by 100. I hope this helps!

Hi there! It's definitely possible for a page to be correctly flagged as duplicate content while a canonical tag is applied. This typically happens in cases where the pages are being seen as duplicates of each other, but are canonicalized to different pages. If you'd like to see a more specific example of where this is occurring on your site, I'd be happy to talk through it with you through a private message to help@moz.com :). I hope this helps to explain!

Hello Olga! Open Site Explorer pulls from our Mozscape index, which updates about once a month. You can always check our update schedule whenever you like! Most new sites and links will be indexed by our spiders and available in Mozscape and Open Site Explorer within 60 days, but some take even longer for a variety of reasons, including the crawl-ability of sites, the amount of inbound links to them, and the depth of pages in subdirectories. Just so you know, here's how we compile our index: We grab the most recent index We take the top 10 billion URLs with the highest MozRank (with a fixed limit on some of the larger domains) We start crawling from the top down until we've crawled 180,000,000,000 pages (which is about 35% the amount in Google's index) Therefore, if the site is not linked to by one of these seed URLs (or one of the URLs linked to by them in the next update), then it won't show up in our index. We update our Mozscape Index every 4 weeks. Crawling the entire Internet to look for links takes 2-3 weeks, but our crawlers are always collecting data. When we need to put the index together, we grab all the data they have collected and start processing, which can take up to 3 weeks to determine which of those links are the most important. Mozscape focuses on a breadth-first approach. This means that we almost always have content from the homepage of websites, externally linked-to pages, and pages higher up in a site's information hierarchy. But deep pages that are buried beneath many layers of navigation are sometimes missed and it may be several index updates until we catch all of these. If our crawlers or data sources are blocked from reaching those URLs, they may not be included in our index (though links that point to those pages will still be available). Finally, the URLs seen by Mozscape must be linked to by other documents on the web, or our index will not include them. I hope this information helps! While the site and links may not be indexed yet, give it some time Thanks! Kevin Help Team

Thanks for the heads up about Open Site Explorer. We are currently looking into this. You can follow the progress of the outage here: http://health.moz.com/ Kevin Help Team

Hi Dirk, Thanks very much for jumping in here! Indeed you're correct. We are experiencing intermittent issues with OSE as a result of longer than normal query times for Mozscape API. We are still keeping an eye and investigating this further at our end, however you're welcome to send further details of any issues you're seeing straight to us at help@moz.com You can also keep up to date with this investigation on our Moz Health page.

I'm having the same problems. I saw that Dave_B posted in March. Now it's September. I've tried to ask about the error message in the chat function but the responses are so slow, it's hardly worth waiting. My question was 53 minutes ago and is still flagged as 'unread'.

We have had this very same thing occur, XXX spammy links, among many others attacks. We found this was a competitor wanting our market share.  They hired kiddy scripters to do the dirty deed of the spammy links which apparently is easy to find people who are always trying to hack XXX sites.  The end results is to ruin your Google backlinking ratings, i.e. in general hurt your overall Goolgle authority and rankings. The secondary result it ties up your time and takes you away from running your business which also is to their advantage. The sad part is that there is no way to protect your self from these low life tactics. Unfortunately, we expeirenced simultaniously robo call attacks taking down our phone systems and over filing our voice mail and at the same time, had every email account know to the general public submitted to spammers to fil up our servers and mail boxes.  Definetly a planned set of events. They topped it off with constant DDOS attacks, and attempted SQL injections and any other thing they could think of. Of course this makes the customers unsure if were still in business because the site was up and down and of course they wondered if it was even safe to shop.  This has gone on for 3 years, until we finally found a safe hosting arrangement, 12 moves later and have extra phone and email security measures in place.   We are still fighting to get back in Google's good graces. Again all taking time away from running your buisness. So a few spammy links to keep up with disavowing, no problem. Count your blessings. (-:

Hello there! Since the http and https URLs are different, the social results will show up different depending on which URL was linked to from that social site - the http URL or the https URL. Since https://docbeans.com/ is the actual URL for your homepage, this should show you the most accurate results in Open Site Explorer. I hope this helps Thanks! Kevin Help Team

Hello again - it's hard to say exactly why a website isn't picked up in our index updates. We are constantly working to improve the way our crawler, Rogerbot, operates, so I'm really hoping that your site is picked up in one of the next updates. For now, the best strategy is to keep working on your link building and watch the update schedule for our next index. Thanks, Kevin Help Team