[htdig] line endings in robots.txt?

Subject: [htdig] line endings in robots.txt?
From: nets@searchtools.com
Date: Thu Dec 02 1999 - 10:38:10 PST

Hi all,

I just saw a report that ht://Dig has trouble with non-PC line
endings in the robots.txt file -- it requires CR/LF.

Is this current? I searched the site and could not tell. If so,
could you add checks for just CR (Mac line end character) and just LF
(Unix line end character)?


To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Thu Dec 02 1999 - 10:51:34 PST