Robots.txt issue with indexation
-
Hello
i have a problem with one of the rules for robots.txt
i have a multilingual mutation of entire page on www.example.com/en/
I want to make indexable /allow/ the main page under /en/
but not indexable /disallow/ everything else under /en/*
Please help me how to write the rule.
-
Name the page you want indexing something and you can use the following:
Disallow: /en/
Allow: /en/index.html
Always test robots.txt in google webmaster tools.
Hope that helps,
Keith
-
There is no option like
/en/index.html
The only adress where you can reach the english main page version is www.example.com/en/
-
Well put the rest of the content in a different directory then and disallow that, thats the only other solution I can think of...