Little confused regarding robots.txt

[DenorL]

Hi there Mozzers!

As a newbie, I have a question that what could happen if I write my robots.txt file like this...

User-agent: *

Allow: /

Disallow: /abc-1/

Disallow: /bcd/

Disallow: /agd1/

User-agent: *

Disallow: /

Hope to hear from you...

[AndersS]

Hi Bimal,

I think you actually don't need the Allow: / portion as it is open by default. So that line can probably be removed. The next 3 lines will disallow all bots to access the directories you have listed.

The last section of your robots txt will block all bots from your entire site, so that should probably not be there unless you really want all bots and spiders to stay out of your site

Some more info can be found here: http://www.robotstxt.org/robotstxt.html

Hope this helps,

Best regards,
Anders

[Alick300]

Completely agreed with Anders Stensones.

[DenorL]

Thanks for your quick response Anders.  Actually, I was thinking the exact about Disallow in last section but not sure. Now it's clear. Your reply helped me a lot.