[htdig] what robots.txt actually does

Subject: [htdig] what robots.txt actually does
From: Kari Suomela (ks@karicobs.com)
Date: Fri Dec 10 1999 - 17:58:45 PST

Friday December 10 1999 11:16, Gilles Detillieux wrote to Daniel

 GD> I think the key sentence that Kari is looking for is this:

 GD> The presence of an empty "/robots.txt" file has no explicit

That's the one! Thx for good responses! :)


To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Fri Dec 10 1999 - 09:58:48 PST