Re: htdig: 3.1.0b2 performance [rebuild|merge]


Geoff Hutchison (Geoffrey.R.Hutchison@williams.edu)
Thu, 05 Nov 1998 19:35:04 -0500


At 4:39 PM -0500 11/5/98, Jeff Breidenbach wrote:
>I have a 25,000 page site indexed with htdig. I add one page to the
>site. I can either rebuild the index from scratch, or merge the new
>data into the database, depending on the command line options I feed
>to htdig/htmerge.

I think you mean using the "-i" flag for htdig to specify an initial dig
and "-a" for both to use .work files. The latter will roughly double the
amount of disk space needed (obviously) but should keep time and memory
constant.

I never use "-i" since it's assumed if there aren't old databases to use.
If you do an "update dig" (i.e. no "-i" and old data), the gain is mostly
in time--it checks every page to see if it's modified first, which saves a
lot of time. Disk space is clearly the same, memory is probably less under
htdig but the same under htmerge.

>PS If anyone is interested, I did see a significant performance
>increase when I upgraded from 3.0.8b2 to 3.1.0b2.

Great!

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:45 PST