Re: htdig: 3.1.b2 -> 3.1.b3 performance degradation +

Michael Pfennich (
Wed, 23 Dec 1998 15:59:45 +0100


> That seems to agree with what Joe Jah said last Thursday. There are
> a couple possible reasons for this, relating to some of the score
> calculation enhancements introduced in 3.1.0b3.
> First of all, there is a memory leak in htsearch, as I mentioned to
> Rodger Zeisler this morning. That could chew up a lot of VM, leading
> to a lot of paging/swapping. Please try this patch and see what impact
> it has on performance.

I have tried this patch for, but the Database I used, was
still from Beta 2,
so I don't know, how accurate my Output is.

> --- htsearch/ Tue Dec 15 10:58:13 1998
> +++ htsearch/ Mon Dec 21 10:11:07 1998
> @@ -852,6 +852,7 @@
> links = 1; // It's a hack, but it helps...
> score += config.Double("backlink_factor")
> * (thisRef->DocBackLinks() / (double)links);
> + delete thisRef;
> }
> thisMatch->setIncompleteScore(score);

In general:
        Memory Usage was dramatically decreased..
        Speed in queries with only a few hits increased..
And I used Symon, to monitor the Disk usage:
A simple Search, over my 70k Doc DB with htsearch Beta 3 -
        -> The Disk was about 40%busy
Wiht Beta 2
        --> The Disk was about 2%busy !!!!!!
--> Both the same query, Output were 6000 Hits

A query with some 70 hits ran in normal time, no differece between beta2
and 3
(even an Boolean)
So I think, that it is ok, as long as there are only a few hits....
At least on my machine, which is an Sun E5000...

What about dynamically switching the new Rankin Order on and off,
on the number of resultsets ???
        --> Cause: Who will view 6000 Result sets ??
                And if you search for an common word, it is good enough to
                know, that you should refine the search

I wish you all a merry Chrismas !!

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:29:56 PST