htdig: Excluding directories and duplicate URLs patch

Joe R. Jah (
Sun, 13 Sep 1998 01:54:44 -0700 (PDT)

Hi Geoff,

Thank you very much for carrying this great software forward.

I compiled/installed ht://Dig 3.1.0b1 a few hours ago on a BSDI 3.1 box.
When I ran the rundig script I realized that the sizes of files in db
directory were dramatically increased, about 70%. I searched several
local file systems and found out that I had many duplicate and triplicate
indexed files. I immediately checked and realized that the

have not been applied to ht://Dig 3.1.0b1; I applied it manually and
recompiled htdig and reran rundig. My databases shrank to their normal
size; no more duplicates;-) Please include this patch in your next


     _/ _/_/_/ _/ ____________ __o
     _/ _/ _/ _/ ______________ _-\<,_
 _/ _/ _/_/_/ _/ _/ ......(_)/ (_)
  _/_/ oe _/ _/. _/_/ ah

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:43 PST