Confirming Robots.txt code deep Directories
-
Just want to make sure I understand exactly what I am doing
If I place this in my Robots.txt
Disallow: /root/this/that
By doing this I want to make sure that I am ONLY blocking the directory /that/ and anything in front of that. I want to make sure that /root/this/ still stays in the index, its just the that directory I want gone.
Am I correct in understanding this?
-
that's right!
Disallow: /root/this/ will bock the complete directly whereas,
Disallow: /root/this/that will only block the "that" within "this"
hope this helps!