Welcome to the Q&A Forum

It would be the best you can do in that situation

Hey Epameinondas, Chris is correct, unfortunately, and it isn't possible to schedule or predict when Rogerbot will crawl a site. The crawl schedule is affected by a number of factors, including when the previous site crawl ended and how many other sites we will be crawling that day, and the schedule may fluctuate from week to week. You can, however dictate how quickly Roger hits your site during the crawl with the crawl delay directive. We recommend having a crawl delay of 10 or lower because Roger will disregard the crawl delay if he can't complete the crawl within our standard service-level agreement. Let us know if you need help with anything else. Chiaryn Help Team Ninja