[htdig] line endings in robots.txt?


Subject: [htdig] line endings in robots.txt?
From: nets@searchtools.com
Date: Thu Dec 02 1999 - 10:38:10 PST


Hi all,

I just saw a report that ht://Dig has trouble with non-PC line
endings in the robots.txt file -- it requires CR/LF.

Is this current? I searched the site and could not tell. If so,
could you add checks for just CR (Mac line end character) and just LF
(Unix line end character)?

Avi

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Thu Dec 02 1999 - 10:51:34 PST