Re: [htdig] Can htdig kill Linux?

Subject: Re: [htdig] Can htdig kill Linux?
From: Max Pyziur (
Date: Wed Dec 06 2000 - 09:28:19 PST

> I just love getting to know new software. There's always some form of
> teething pain. Yesterday, I started running my first set of reasonably
> large htdig/htmerge processes. Came in today to find the Linux server
> (which is running nothing besides basic Mandrake processes and, of
> htdig) was deader than a doornail (have to say "deader than" because
> "hung more than" would just be too weird).
> In any case, I couldn't telnet into the Linux box, couldn't run my KDE
> console, nada.

Can you check your system logs? Were there any messages on your console?
Did your machine restart ok?
W/o any of that it sounds remotely like a "too many files open" problem.

> I've never seen Linux hang like that before. Almost makes me wish for

> But the net of it is this: is htdig liable to do this? Can I count on
> htdig in a production environment or do I need to go back to square one?
> What's your experience? Advice?

We have about 15,000 pages on our primary site, 6,000 on one of our
virtual hosted sites. The site is in two languages - English and
with some items in a third, Russian. The locale is set for uk_UA.cp1251.

Our machine is currently two P3-550s, 512MB RAM, 2 20GB HDS; our
os is RH6.2 with most if not all of the updates.

We rebuilt htdig rpms so that it would install to peculiarities of
our filesystem.

Indexing each site takes about twenty to thirty minutes. Admittedly, we
got the firewpower (CPUs) to do that.

> Thanks in advance,
> David

Max Pyziur BRAMA - Gateway Ukraine

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.
List archives: <>
FAQ: <>

This archive was generated by hypermail 2b28 : Wed Dec 06 2000 - 09:37:35 PST