AW: [htdig] htdig says "no server running" after a while


Subject: AW: [htdig] htdig says "no server running" after a while
From: Michael Balzer (balzer@bbcomp.de)
Date: Wed Dec 22 1999 - 16:14:16 PST


hi geoff,

> Are you indexing a server on the same box? In that case, I
> highly suggest
> using local_urls (and company) to skip using HTTP. This will
> speed things
> up considerably as well as cutting down the load on Apache.

i'd try that, but unfortunately some parts of the
document database are generated on the fly, so file
access won't do.

> > it *seems* the error occurs the later the more apache processes
> > are running. but the current run is already going with 20
> > (!) apaches, which is more than ever needed on our internet
> > server... and i'm not yet sure it will index everything now.

update: the last run (with 20 servers) has gone through
the database completely. everything is fine now ;-).

the htdig index sums up to about 1.3 gb. a query is
normally answered within 3-4 seconds (this is a
celeron 333 system with 128 megs of ram). i like htdig.

> I'm not sure why you're getting this, but you might try
> running htdig with
> -vvv or higher, redirecting that into a log and taking a look
> at the HTTP
> headers it gets back. This way you can see what Apache is telling the
> indexer.

i'll try -vvv if the problem occurs again. but it did
not look like htdig got anything back from the server
for those documents. i really think htdig has a timeout
problem with apache if there are not enough servers to
handle requests fast enough. there's a certain delay
if apache has to spawn a new process, especially if
memory is tight, maybe that's the point where htdig
gives up too early.

regards,
michael balzer

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Wed Dec 22 1999 - 16:28:39 PST