Robots.txt question

[seoug_2005]

I want to block spiders from specific  specific part of website (say abc folder).

In robots.txt, i have to write -

User-agent: *

Disallow: /abc/

Shall i have to insert the last slash. or will this do

User-agent: *

Disallow: /abc

[RyanKent]

Use the trailing slash.

More about robots.txt can be learned at this site: http://www.robotstxt.org/

The trailing slash indicates you are blocking a folder. Without the slash the object would be considered a file (i.e. page). I am not sure what the result would be if you tried to block a folder without the trailing slash. Even if it worked it would not be the correct code and may lead to various bots treating it differently.

[Entrusteddev]

Hi Atul,

Add the trailing slash.

/abc could be a page url. Where as /abc/ is definitely a folder.

http://www.robotstxt.org/robotstxt.html <-- Everything you ever wanted to know about robots.txt

Regards

Aran

[EDIT: Damn it, Ryan submitted whilst I was answering! Must type faster ]

[RyanKent]

EGOL does it to me all the time!

[Entrusteddev]

at least our answers agree, so no Atul is doubley sure of how to do it...

[seoug_2005]

Thanks for clearing my doubts.

[seoug_2005]

I am the same person with the username seoug, but lost that account. So, had to start afresh !  I was a PR0 member, but accidently deleted that account ( it was not intentional ). And now , when i tried login in, i get a message that seoug name is already taken.

[RyanKent]

If you contact the help desk, they can probably help you get your old account back.

[seoug_2005]

I will do so. And hope to get that back.