Is there a relationship between Rogerbot hits and Invalid Traffic?

[zouhour.halawi]

This post is deleted!

[tawnycase]

Hey there! 
Tawny from Moz's Help Team here.

1. Our crawler crawls all the links on the site it can find, which may include banners.
2. Our crawler is based in Dallas, but it should still crawl sites located internationally, including for sites in Saudi Arabia and United Arab Emirates.
3. We've not noticed this issue before, and unfortunately, we don't have a good way of slowing down our crawler's crawl speed. Sorry about that! Typically you would be able to slow down a crawler with a crawl delay in a robots.txt file, but our crawler doesn't currently react to crawl delays, and so those can't be used to slow down our crawler.

​Sorry I can't be much help here! If you still have questions, feel free to reach out to us at help@moz.com and we'll do our best to answer all your questions for you.

[zouhour.halawi]

Hi Tawny,

Appreciate your prompt feedback!

I still have 3 questions though:

1- If I am to disable Rogerbot, I assume I would lose all the data related to "site crawl" feature + "page optimization" feature. But Does it mean as well I would lose all the data related to "keywords Ranking"? Plus what other data would I be compromising ?

2- can I set up the bot to crawl the site once a month, instead of weekly?

3- Is there a way to block Rogerbot from crawling our ads?

Appreciate your help.

Thx

Zouhour

[tawnycase]

Hey Zouhour,

1. Blocking Rogerbot would make it so that our crawlers could not access your site to offer Site Crawl or Page Optimization data, but your keyword rankings should still be unaffected. We don't need to be able to crawl your site to report on your keyword rankings. Only the data that requires our crawler to reach your site would be compromised, including Site Crawl, Page Optimization, and Crawl Tests.

2. Unfortunately, there's no way to schedule our crawler — it crawls once per week, based on when you set up your Moz Pro Campaign. There's no way to change this crawl schedule.

3. If you were able to set up a Disallow directive in your robots.txt file to exclude your ads, then you could stop Rogerbot from crawling them. That will only work if there's a distinct URL pathway for your adds that will allow you to set up a Disallow directive for our user-agent, Rogerbot. You can read more about robots.txt files and best practices over here: https://moz.com/learn/seo/robotstxt

Hope that helps! If you've got any more questions, feel free to give us a shout at help@moz.com and we'll do our best to help you out!