Can't find mistake in robots.txt
-
Hi all,
we recently filled our robots.txt file to prevent some directories from crawling.
Looks like:
User-agent: * Disallow: /Views/ Disallow: /login/ Disallow: /routing/ Disallow: /Profiler/ Disallow: /LILLYPROFILER/ Disallow: /EventRweKompaktProfiler/ Disallow: /AccessIntProfiler/ Disallow: /KellyIntProfiler/ Disallow: /lilly/now, as Google Webmaster Tools hasn't updated our robots.txt yet,
I checked our robots.txt in some ckeckers.They tell me that the User agent: * contains an error.
**Example:****Line 1: Syntax error! Expected <field>:</field> <value></value>
1: User-agent: *****`I checked other robots.txt written the same way --> they work,`**accordign to the checkers...**`Where the .... is the mistake???`** ``` -
Hi!
The robots.txt is fine. Some checkers return wildcards as an error, as not all crawlers supported "*". I wouldn't worry about it

-
_Hi, _
_Just wondering .. Did you save the txt file in ANSI format? Sometimes, people mistakenly save it different format and this is where the problem creeps in. _