[htdig] [ANNOUNCE] Release of ht://Dig version 3.1.4


Subject: [htdig] [ANNOUNCE] Release of ht://Dig version 3.1.4
From: Kari Suomela (ks@karicobs.com)
Date: Fri Dec 10 1999 - 16:59:08 PST


Thursday December 09 1999 19:00, Geoff Hutchison wrote to All:

 GH> that it's not limited to robots.txt parsing.

I've tried to figure out what robots.txt actually does, and how to
properly configure it. I have read the FAQs etc. on several sites, but
cannot find plain language explanations. Apparently some searches fail,
and others can't find my server at all because of this.

Could someone, please, explain what to do with that file, so all search
engines and agents would work ok.

 KS

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Fri Dec 10 1999 - 08:59:11 PST