[htdig] Slow Initial Dig.

Lee Mathers (ml_htdig@help.nt.net)
Tue, 26 Oct 1999 14:12:49 -0400

Well I just recently upgraded from to 3.1.3 from 3.1.2. I'm running a K6
3 /300 with 128 MB of Ram and 450MB of Swap. Linux Kernel 2.2.12 and
Hacked up Version of Debian 2.1

This is the first time I've run into this problem. I can't build an
initial database. htdig -i -s -u xx:xxxxxx -c /opt/gene/conf/index6.conf
has been running now for over 72 Hours and has not really built a big
database. It also hasn't even begun to actually hit the web server aside
from grabbing the initial index.html file which is BTW 17Mb.
57899008 Oct 25 17:28 db.docdb
50216923 Oct 26 04:17 db.wordlist

In comparision when I was using the previous Version of htdig 3.1.2 with
the same command line options and same conf file it would take 12 or 13
hours to do a complete index of the site. with total db sizes of.

184994816 Aug 31 10:11 db.docdb
16670720 Aug 31 10:11 db.docs.index
  912384 Aug 31 10:29 db.metaphone.db
  771072 Aug 31 10:29 db.soundex.db
57354120 Aug 31 09:17 db.wordlist
46577664 Aug 31 09:17 db.words.db

So If someone could help me out with this one by suggesting to be what I
might have buggered up I would be very happy. As it seems very strange
to me that after running for 3 days solid it hasn't even hit the web
server 2 times.
As I finish writing this the htdig process is using 251Mb of Ram which
includes about a 140Mb of Swap. And it's been that way for about a day.
(Yes the process isn't a zombie or anything like that. Top reports that
it's still running and using more or less CPU (avg around 5%)

To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Tue Oct 26 1999 - 11:22:08 PDT