[htdig] Indexing a list of sites -- catching failures


Subject: [htdig] Indexing a list of sites -- catching failures
From: twallace@neo.lucidgreen.com
Date: Fri Aug 25 2000 - 20:05:16 PDT


I have set up htdig so that every night, it indexes a long list of small
web sites. In general, this works very well, but I've found that I have to
be very careful when adding new sites to the end of the list.

Some sites seem to cause htdig to fail. When this happens, htdig doesn't
continue on with the rest of the list -- it simply skips to the next step
in rundig. This means that I have to do some careful adding and substracting
to the list of sites before I figure out what caused the index to fail.

What I would like to do is to somehow index each site separately and have
some kind of error log if htdig hits a site that it fails on (for whatever
reason). Then, I would like for it to procede to the next site in the list,
whether or not it failed to index the previous site.

Thanks for any help,
Todd Wallace

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Fri Aug 25 2000 - 13:05:20 PDT