Is there a way to change crawling time for moz and to do it in a less demanding hour of the day?
-
Moz now crawls during the morning. Would it be possible to crawl during the evening?
-
I don't think that's possible, Epameinondas, but you can slow rogerbot down when it comes to crawl your site by adding a crawl delay directive to your robots.txt file:
User-agent: rogerbotCrawl-delay: 1
More info on the crawl delay directive:
-
Hey Epameinondas,
Chris is correct, unfortunately, and it isn't possible to schedule or predict when Rogerbot will crawl a site. The crawl schedule is affected by a number of factors, including when the previous site crawl ended and how many other sites we will be crawling that day, and the schedule may fluctuate from week to week.
You can, however dictate how quickly Roger hits your site during the crawl with the crawl delay directive. We recommend having a crawl delay of 10 or lower because Roger will disregard the crawl delay if he can't complete the crawl within our standard service-level agreement.
Let us know if you need help with anything else.
Chiaryn
Help Team Ninja