Excluding Directories from RogerBot

[TonyG5003]

I need the ability to exclude directories (parameters, etc.) from within crawl settings of some sort - other scanning programs provide this flexibility.

I realize that one can simply edit the Robots.txt file - but many SAAS website platforms do not allow access to the robots file (if there is one) - or it's limited - meaning the client must pay for customizations to edit the file.

Unfortunately, for many clients, I cannot use the scanning tools at Moz because of this. Does anyone know if there plans to add exclusion options directly within the scanning tools?

Thanks!

[DavidLee]

Hi Tony

I'm afraid the only way to block rogerbot will be through robots.txt as we do not have a way to customize our crawler. It would definitely be a great feature request for our product team if you would like to provide any suggestions or ideas directly to them at https://seomoz.zendesk.com/forums/293194-seomoz-pro-feature-requests

Hope this helps!