Moz crawler

[Emanuele_Ricci]

I have a site which is in a non production status.

Crawlers are blocked vis robot txt.

User-agent: *
Disallow: /

I WANT TO MAKE A CRAWLING TEST WITH MOZ CRAWLER (RogerBot) ,
how can I allow your crawler to get in and prevent other crawlers from indexing the site?

Thanks

memok

[GPainter]

did you take a look through - http://moz.com/learn/seo/robotstxt

I'll give you a clue -

Allow a specific web crawler to visit a specific web page

Disallow: /no-bots/block-all-bots-except-rogerbot-page.html User-agent: rogerbot Allow: /no-bots/block-all-bots-except-rogerbot-page.htmlmore info -http://moz.com/help/pro/rogerbot-crawler&http://moz.com/help/pro/what-is-rogerbot-

[maxweb]

Try this:

User-agent: *
Disallow: /
User-agent: rogerbot
Allow: /