htdig: Excluding directories and duplicate URLs patch


Joe R. Jah (jjah@cloud.ccsf.cc.ca.us)
Sun, 13 Sep 1998 01:54:44 -0700 (PDT)


Hi Geoff,

Thank you very much for carrying this great software forward.

I compiled/installed ht://Dig 3.1.0b1 a few hours ago on a BSDI 3.1 box.
When I ran the rundig script I realized that the sizes of files in db
directory were dramatically increased, about 70%. I searched several
local file systems and found out that I had many duplicate and triplicate
indexed files. I immediately checked Retriever.cc and realized that the
patch

   ftp://sol.ccsf.cc.ca.us/htdig-patches/3.0.8b2/Retriever.cc.0

have not been applied to ht://Dig 3.1.0b1; I applied it manually and
recompiled htdig and reran rundig. My databases shrank to their normal
size; no more duplicates;-) Please include this patch in your next
release.

Joe

     _/ _/_/_/ _/ ____________ __o
     _/ _/ _/ _/ ______________ _-\<,_
 _/ _/ _/_/_/ _/ _/ ......(_)/ (_)
  _/_/ oe _/ _/. _/_/ ah jjah@cloud.ccsf.cc.ca.us

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:43 PST