Roger Bot

[Bush_JSM]

Hi Mozzers,

I have a dev site that I want to run your crawl text on (Roger Bot) but I want to ensure the other engines don't crawl it.

What is the Robots.txt line that I need to make sure only Roger bot can get in and not Google etc?

Please advise

Thanks

Gareth

[intSchools]

User-agent: *

Disallow: /

User-agent: rogerbot

Allow: /

[kyleNeedham]

HI Gareth, Your robots.txt should look like this; User-agent: * Disallow: / User-agent: rogerbot Allow: /