Re: [htdig] ht://Dig 3.2.0b1 and 3.2.0b2-022000 Extremely Slooooow


Subject: Re: [htdig] ht://Dig 3.2.0b1 and 3.2.0b2-022000 Extremely Slooooow
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Wed Feb 23 2000 - 06:31:22 PST


First off, I'm not sure an RPM version would have any idea about a
non-rpm version. That goes in general, not just for ht://Dig. As a
general rule, I'm also pretty cautious about removing old versions in
favor of new ones, especially "beta" software. We marked 3.2.0b1 as a
beta for a reason (it hasn't been tested a whole lot) and while I
doubt anyone will get burned using it, as the announcements said,
it's not recommended for production use.

Now, as far as the speed of indexing in 3.2.0b1 (and current
snapshots), I probably need to make this a FAQ. Right now, it's
probably not going to be faster than 3.1.x versions and is quite
likely to be slow. We rewrote the whole layout of databases and in
the process made quite a few trade-offs against the indexer.

For example, the previous version stored documents by URL, which made
indexing fast but searching slower. But I'd much rather have a slower
indexer if it means my searches will be faster! Similarly, previous
versions required sorting the words in htmerge, which meant the
indexer didn't have to bother storing them in any organized way. This
version doesn't need to sort (soon it may not need htmerge) but it
slows down the indexer.

But the important thing to remember is that these are *betas*--we're
looking for feedback. We'd love to have accurate performance and
requirement feedback. The new database layout is probably going to
require more disk space (especially if compression is off), but you
won't need as much memory for htmerge. So hard numbers would be
wonderful. This will help us target what needs improvement. Further,
if anyone wants to help improve indexing performance, I'm sure we can
come up with a list.

(I hope that answers your question.)

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Wed Feb 23 2000 - 06:48:43 PST