[htdig] what robots.txt actually does


Subject: [htdig] what robots.txt actually does
From: Kari Suomela (ks@karicobs.com)
Date: Fri Dec 10 1999 - 17:58:45 PST


Friday December 10 1999 11:16, Gilles Detillieux wrote to Daniel
MacKay:

 GD> I think the key sentence that Kari is looking for is this:

 GD> The presence of an empty "/robots.txt" file has no explicit

That's the one! Thx for good responses! :)

 KS

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Fri Dec 10 1999 - 09:58:48 PST