Welcome to the Q&A Forum

Might've gone a little endorse-happy there, but the three of you hit all the points I was going to make.

Hi Aman!

This won't necessarily solve your problem, and I sort of hate just plugging a Moz tool, but have you ever tried running a Full SERP Analysis Report in the Keyword Difficulty tool? It can let you know what factors are helping your top 10 SERP competitors rank.

There's a great video on using it here.

There's nothing planned at the moment, but it's definitely something we're aware of.

Your best bet may actually be to run a Crawl Test using this tool. It'll crawl up to 3000 pages of any given domain, and report back in much the same way as the weekly crawl. That one should still run as scheduled.

Hi Jay,

I definitely sympathize, and I'm sorry you're dealing with this. I'm aware that there is a small subset of hosts that feels our crawlers are too aggressive, and that yours isn't the only one. (As it happens, though, both of those posts are regarding Dotbot, the Mozscape Index crawler, and the first response to the Q&A post indicates that the host has no issues with Rogerbot.)

Our challenge in this area centers around our need to accommodate a vast and diverse customer base. We have customers with sites spanning millions of pages, and we're obligated to meet our service-level agreement with them to provide data in a timely manner. As it is, many of our larger customers receive crawl updates only once per month in order to prevent Rogerbot from having to crawl too aggressively. We've found that the rate at which Rogerbot crawls is acceptable to the vast majority of hosts, and that the few who would prefer a less-aggressive crawl are almost always willing to apply a crawl limit.

This is especially true given that Rogerbot only crawls sites on-demand, either as part of an ongoing Moz campaign or in a Crawl Test. Since having a site crawled by Rogerbot is voluntary, it generally falls on that few to adjust their crawl limits accordingly. We simply can't adjust our crawl rate to suit the requirements of that few. This is the case with all of our competitors, as well.

That said, I'd still love to show a server log from one of those old crawls to our engineering team. If something _is _amiss with our crawler, we absolutely want to make sure it's addressed. I understand you've been in touch with our Help team, so you can go ahead and send it over to them.

I'm sorry that Rogerbot crawls too aggressively for your host. It's designed to crawl as aggressively as necessary in order to complete the crawl in a reasonable amount of time to keep you from having to wait too long for your campaign data.

Since Rogerbot was already blocked by the time the new robots.txt was implemented, you may want to see if your host would un-ban the crawler and test whether it follows the limit.

All that said, if Rogerbot _is _crawling in a way we don't intend, we'd like to check it out. If there's any way you could send me a server log from one of those old crawls we could investigate.

It's never really been a problem, and Rogerbot SHOULD respect crawl limit. May I ask, when did your host block the crawler?

EDIT: Also, it looks like the robots.txt file was massively changed pretty recently on at least one of your campaign sites. Do you know when that was? Feel free to private message me.

Hi there!

You're definitely right that those tools have trouble with non-Latin characters.

There are still other tools in Moz Pro, though, that you should be able to get some value from Crawl Diagnostics should be pretty useful, in particular. I'd recommend this Help Hub article—actually, you'll likely find the Help Hub useful, in general—and this video to get you started.

It's definitely possible to take care of everything in On-Page Optimization/Grader manually. You should get a solid handle on on-page targeting by reading through the Beginner's Guide to SEO. After that, How to Rank will give you a really solid picture of a successful organic campaign. Keep in mind that keyword targeting—which is what those tools help with—is only a small portion of the factors that will help you rank.

I admit that I'm not really aware of other tools that do the same thing with non-English characters, though. Hopefully someone will know of a few!

Hi Alan,

I'm really not a Wordpress expert, so I can't comment much on implementing a rel="canonical" tag using their interface. There's some good general information on canonical here, though: http://moz.com/learn/seo/canonicalization

If you're set on solving this with a canonical, I imagine you'd point www.nyc-officespace-leader.com/601-west-26th-street-starrett-lehigh-building-contains-executive-office-space-manhattan/page/3 to www.nyc-officespace-leader.com/601-west-26th-street-starrett-lehigh-building-contains-executive-office-space-manhattan. I'd include the full URL.

Out of curiosity, what purpose does having both pages actually serve? The content is pretty similar on both.

I'm afraid Jonathan is right—if Rogerbot is the crawler that gathers data for the crawl reports, so if he can't crawl your site you won't get crawl data. Your only other option would be to use a crawl tool other than Moz.

Hi All!

It looks like Dev Dan asked a follow-up question here, if anyone can help him out.

http://moz.com/community/q/gtm-migration-from-old-to-new-verison-as-old-version-closing-on-1st-april

Hi there!

Just for a bit of clarification, is Webmaster Tools alerting you to duplicate title tags, duplicate meta descriptions, both, or duplicate page content overall? I think that'd make it a little easier to answer your question.

Not really my area of expertise, I'm afraid. Just making sure that folks who'd be able to help can see the conversation already going on in the other thread.

Hi Rich!

Another thing to keep in mind is that listings don't necessarily update as soon as you make changes in Moz Local. Depending on the partner, it can take anywhere from a few days to a few weeks for the changes to be made. The issue has to do with how each directory updates their listings.

Hope that helps a bit!

Hi there!

I think it might help a bit if we knew a bit more about what you're asking. Perhaps more information about the overhaul, and exactly what kind of help you're looking for?

Thank you!

Looks like this is being handled in this thread:

http://moz.com/community/q/google-plus-company-page-verification-2

Joe's suggestion is awesome, and I agree completely! I'd add Cyrus Shepard's "How to Rank," though, which really does a good job of laying out a successful SEO strategy.

http://moz.com/blog/how-to-rank

Hi Brad!

Donna is spot-on; I'm especially a fan of Screaming Frog. That said, it looks like you're a Moz Pro subscriber, so you can use your Moz Analytics campaigns to crawl any site up to 50,000 pages.

Whenever you set up a new campaign, it automatically crawls the site it's set to track. It then updates every week, with the crawl results surfaced in Crawl Diagnostics. It looks like your account can hold up to 5 active campaigns, but they can always be deleted and more can be added.

Does that help at all?

I agree with everyone here (thanks, guys!), and I want to add that DA and PA are both most useful when benchmarking against competitors, not as an absolute value.

Take this scenario—a high-authority industry site links to a large number of other sites in the same industry, and they all benefit from its authority. But if for whatever reason that site goes down or loses authority, all sites it links to could be affected. That could conceivably bring down the DA of a major portion of the vertical.

None of those sites did anything wrong, and since they're each others' main SERP competitors, their rankings may not change at all. But they'll now all have lower DA.

Does that help at all?

Hi there! I saw your tweet @Moz, so I wanted to follow up. You can get the guidelines for CSV uploads—including business hours—here:

https://moz.com/local/help/documentation

That said, it sounds like the answer to this might be a little more technical, and thus a job for our illustrious Help Team. The question is in their queue, which generally means there'll be a response within 24-48 hours. Sit tight!