Welcome to the Q&A Forum

Hi Aimee, Dave here from the help team

I took a look at your account and I see a campaign called JHM that only has two pages crawled, so I am assuming that is the site in question.  The two pages are the domain and then the redirect to the WWW version.  When I load the last page we crawled and then look at the source code, I noticed that the site is entirely javascript and does not contain any other HTML links.  Since our crawler follows HTML link structure, we are hitting this page and then stopping when we don't find anything else to crawl.  Hopefully that helps to clarify, let me know if you have any other questions about this or feel free to reach out to us at help@moz.com.

Hey!  Dave from the Help Team here,

we are currently running some tests internally to try and determine what is the optimal length now for Meta Descriptions.  Once we have some conclusive results we will update our tools.  We should have a blog post coming out soon about it, so stay tuned!

Hey Lee, Dave from the Help Team here.

Site Crawl will detect any error that we report on, for any page that is accessible to the crawler.  As you develop and change your website, new crawl issues might get detected as well.  Hopefully that helps to clarify, if you need us to look into specific examples, feel free to email help@moz.com.

STANDARD level subscription crawl limits have recently been changed to 20k pages per tracked site allowing for 5 tracked sites with a total of 100k pages.  MEDIUM level plans still crawl up to 50k per tracked site.  You can see what each level subscription gets on our pricing page.

Hey, Dave from the Help Team here!

Unfortunately Moz does not have any type of API tool similar to On Page Grader, but that is a good idea.  I will pass along your feedback to our product team.  Hopefully that helps!

Hey Bill, Dave here from the Help Team!

If you are finding these in Site Crawl, you can also find the referral link, which is where we found these 404 links.  Using that referral link might help you track down where in the source code these links are being found if you wanted to remove them.  Hopefully that helps to clarify!

Hey Damien, Dave from the Help Team here!

Unfortunately at this time Keywords By Site is specific to Google US.  We hope to be adding more international search data soon!

Hey, Dave here from the Help Team!

This is really up to you and how you want to organize your campaigns, however I think you are right about not overly complicating it.  I would suggest that you track your entire domain in one campaign and categorize your keywords with labels.  If you decide to later track specific sections of you site, you can, but its probably good to have one campaign with an overall view of your entire site.  Hopefully that helps!

Hey Ben!  Dave here from the Help Team,

Its hard to say what could be going on with these links without being able to look at your website.  Would you be able to write into help@moz.com with this info so we can take a deeper look?

thanks!

Hey Jeannie, Dave from the Help Team here.

Sorry about the trouble!  Typically to resolve this we'd recommend adding a crawl delay for rogerbot to your robots.txt file.

Unfortunately, at this time, our crawler isn't able to follow crawl delay directives in robots.txt files.  I'm so sorry about that!  Our engineering team is aware of the issue and are working towards a resolution however I don't have an ETA at this time.

One possible solution in the interim while we work towards a fix here would be to archive the campaign so we stop crawling it, and then when we have the ability to follow crawl delays, you could un-archive your campaign. You could also block rogerbot altogether in your robots.txt file - it would look like this:

User-agent: rogerbot
Disallow: *

I am going to show this conversation to our development team as well. Again, sorry for any inconvenience - please do let me know if there's anything else I can help with!

Hey Cesare, Dave here from the Help Team.

We are working on increasing better functionality around international URLs/Keywords for Keyword By Site, but as for right now its mainly U.S. focused.  Sorry for that inconvenience that causes!

Hey!  Dave here from the Help Team.

It looks like we are getting a 503 status code when trying to access your robots.txt file.  You will want to check your server logs for the response from User-agent: rogerbot/1.2 (http://moz.com/help/pro/what-is-rogerbot-, rogerbot-crawler+aardwolf@moz.com) and make sure you are not blocking us from accessing your site!  Hope that helps

Hey, Dave from the Help Team here.

You could track the entire site with all your keywords listed and label them per variation, or you could setup separate campaigns restricted to the subfolders of the variant languages and track the multilingual keywords in each perspective campaign.  As long as the keywords are latin characters, you shouldn't have an issue.  Hopefully that helps!

Hey, Dave from the Help Team here!

I just ran your site through our crawler and didn't have any issues crawling your site.  I think the Domain Authority score of 1/100 is actually caused by us not having your site in our Mozscape Index.  To find out more about how we Index the web, you can read about it here

https://moz.com/help/guides/research-tools/open-site-explorer/updates/index-process

Hopefully that helps to clarify, let us know if you need anything else!

Hey!

Sure you can write into help@moz.com and we will get you sorted out!

Hey, Dave from the Help Team here

Are you receiving this error when running your site through Open Site Explorer?  Can you please provide the URL that you are checking?

thanks!

Hey, Dave here from the help team!

On Page Grader is uses DNS checks to validate and it looks like all of your nameservers are not valid.  You may want to start looking there.  I also responded to your help ticket, if you have any follow-up questions, feel free to hit me back there!

https://www.screencast.com/t/jmKQ5BFuN8

I also responded to your help ticket, if you have any follow-up questions, feel free to hit me back there!

Hey!  Dave here from the Help Team,

There are a couple different things you can do to mark items that you have done.  One of the new features we have implemented into Site Crawl is the ability to mark items as "Fixed".  This tool can be handy if you know that you have fixed issues with your site but are still waiting for your next update.  Another trick you might want to do is download your "all crawled pages" CSV and then create a "notes" column.  It wont live in the Moz dashboard but at least you would have a good record!  Hopefully those options help you out!

Hey!  Dave here from the Help Team,

We are aware that this is something our customers want and we keep track of everyone that writes in about it but at this time we cannot guarantee this functionality or have a valid timeframe for it.

Hey!  Dave here from the Help Team,

If you are tracking the site in a campaign, you can navigate to the Site Crawl section and look at the "all crawled pages" report.  This will give you the status codes, referral links, crawl depth, etc.  That should help point you in the right direction!