Re: htdig: 3.1.0b2 performance [rebuild|merge]

Geoff Hutchison (
Thu, 05 Nov 1998 19:35:04 -0500

At 4:39 PM -0500 11/5/98, Jeff Breidenbach wrote:
>I have a 25,000 page site indexed with htdig. I add one page to the
>site. I can either rebuild the index from scratch, or merge the new
>data into the database, depending on the command line options I feed
>to htdig/htmerge.

I think you mean using the "-i" flag for htdig to specify an initial dig
and "-a" for both to use .work files. The latter will roughly double the
amount of disk space needed (obviously) but should keep time and memory

I never use "-i" since it's assumed if there aren't old databases to use.
If you do an "update dig" (i.e. no "-i" and old data), the gain is mostly
in time--it checks every page to see if it's modified first, which saves a
lot of time. Disk space is clearly the same, memory is probably less under
htdig but the same under htmerge.

>PS If anyone is interested, I did see a significant performance
>increase when I upgraded from 3.0.8b2 to 3.1.0b2.


-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:45 PST