[htdig] htdig 3.2b2 performance

Subject: [htdig] htdig 3.2b2 performance
From: Ravindra Wankar (rwankar@iname.com)
Date: Sun Jun 11 2000 - 10:38:23 PDT

Phrase match seems very very slow (as compared to "all words" and "any

- RedHat 6.1 on 300Mhz Celeron with 128 MB RAM, IBM 10GB IDE disk (5400
RPM) 1MB cache.
- 1186 HTML pages total size of ~4MB

Also, when running htdig, initially htdig takes up 97-98% of CPU time.
Memory usage is high but I don't see swapping. After a while the cpu
usage drops to around 40%. Mem is still fine.

Similarly when htsearch is run I see almost 90-95% CPU usage. What
happens if there are 10 simultaneous searches?

Would moving to MYSQL DB help? I don't see a patch for 3.2 versions.

Does anyone know what is/are the bottlenecks? Disk/Mem/CPU? e.g. given
the above configuration, what can be changed to speed things up?

Also, how do I check I/O activity on linux?

-- Ravi.

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Sun Jun 11 2000 - 08:26:29 PDT