Questions
-
Rogerbot will not crawl my site! Site URL is https but keep getting and error that homepage (http) can not be accessed. I set up a second campaign to alter the target url to the newer https version but still getting the same error! What can I do?
It will probably help if you delete the following line from your robots.txt file: User-Agent: rogerbot Disallow: / Dirk PS You also block /js/ for all bots - Google doesn't really recommend that
Other Research Tools | | DirkC0