Re: [htdig] Indexing a list of sites -- catching failures


Subject: Re: [htdig] Indexing a list of sites -- catching failures
From: christopher.murtagh@wcg.mcgill.ca
Date: Tue Aug 29 2000 - 11:14:47 PDT


On Tue, 29 Aug 2000 D.J.Adams@soton.ac.uk wrote:

>>Todd Wallace wrote:
>>
>> Some sites seem to cause htdig to fail. When this happens, htdig doesn't
>> continue on with the rest of the list -- it simply skips to the next step
>> in rundig. This means that I have to do some careful adding and substracting
>> to the list of sites before I figure out what caused the index to fail.
>
>I do a weekly index of over 900 servers and have never had this happen.
>What version of htdig are you using?

 I've had a similar problem. I have a list of around 90 servers, and if
one of them times out (or in this case not time out and fail to connect),
the whole process can come to a stand-still. The only solution I've found
is to remove it from my start.url file. I'm running 3.1.5 on PPCLinux
(YellowDog).

 Some sort of timeout adjustment would seem to be needed. Anyone else
exprience this?

Cheers,

Chris

-- 

Christopher Murtagh Webmaster / Web Communications Group McGill University Montreal, Quebec Canada

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Tue Aug 29 2000 - 11:14:15 PDT