Robots.txt query

[Karen_Dauncey]

Quick question, if this appears in a clients robots.txt file, what does it mean?

Disallow: /*/_/

Does it mean no pages can be indexed? I have checked and there are no pages in the index but it's a new site too so not sure if this is the problem.

Thanks

Karen

[William.Lau]

What is the user agent?

[Karen_Dauncey]

User-agent: *

Thanks for your response.

[William.Lau]

That blocks all spiders from viewing those pages. I am not sure what and who did the /* /_/, but unless there is something there they don't want indexed then it is not necessary to keep it.

One thing you mind want to keep in mind as well, just because you block it on robots txt, doesn't mean a spider can't still go there.

Sometimes they don't listen to the robots txt(looking at you baidu)

[Karen_Dauncey]

Thank you so much, that is a great help!