Re: [htdig] Capacities/Cababilities of htdig

Geoff Hutchison (
Wed, 3 Feb 1999 14:17:55 -0500 (EST)

On Wed, 3 Feb 1999, Aviram Carmi wrote:

> - How many sites/pages can it index in a reasonable amount of time?
> - How many user queries can it handle?
> - What is the largest number of sites/pages that you index using htdig?

See"> for one

I now have 75,000 URLs on my PPro 100 box with 128MB RAM. It returns
searches in around 5-10 seconds as well (depending on number of matches,
naturally). It takes about 5 hrs to index and merge completely using the
current development code and about 1/2 hr to do updates after that.

Since the load never seems to go very high, I would expect it to easily
handle many concurrent user queries. I've never done much testing of that.

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Wed Feb 10 1999 - 17:09:05 PST