Subject: Re: [htdig] 3.1.5 -- Limiting Duration / Resources for HTDIG
From: Gilles Detillieux (firstname.lastname@example.org)
Date: Wed May 24 2000 - 12:00:20 PDT
According to Sphboc@aol.com:
> I am running into some situations where HTDIG appears to be going into an
> infinite loop. The pattern is that it either does, or does not, occur when
> searching a specific website; re-attempting the HTDIG--against the same
> site--usually produces the same result.
> Is there some straightforward way to limit the number of bytes which can be
> written into either wordlist or the log file; to ensure unconditional
> termination of processing once this limit is exceeded? (In practice, HTDIG
> is invariably still producing output when it's externally terminated; I
> haven't run into any instances of looping without producing output).
There's a server_max_docs config attribute that will limit the number of
documents htdig will index per server. However, you may want to run htdig
with one or more -v options to get some feedback on what it is doing on the
servers that seem problematic. It might give you some hints as to how you
can tweak your exclude_urls and limit_urls_to attributes to avoid indexing
certain branches on these servers that you'd rather avoid.
See http://www.htdig.org/attrs.html for descriptions of all these attributes.
-- Gilles R. Detillieux E-mail: <email@example.com> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930
------------------------------------ To unsubscribe from the htdig mailing list, send a message to firstname.lastname@example.org You will receive a message to confirm this.
This archive was generated by hypermail 2b28 : Wed May 24 2000 - 09:48:57 PDT