Re: [htdig3-dev] fault tolerance

Subject: Re: [htdig3-dev] fault tolerance
From: Geoff Hutchison (
Date: Wed Feb 23 2000 - 07:33:39 PST

At 11:53 AM +0200 2/23/00, Valdas Andrulis wrote:
>I would like to ask about fault tolerance of htdig. thhe problem is
>that, when i index many documents(as i already said in previous letter)
>htdig crashes, and htmerge after that says that is only 72 documents and
>0 words, so all the work is lost? or is there a way to repair those
>databases ar get good data from broken databases?

I don't know--you could try to use the htdb_recover program in the
htdb directory, but you might want to see below. BTW, if you've run
htmerge at this point, you're unlikely to recover more information
from them. The purpose of htmerge is to put the databases in a nice
consistent state, so it most likely tried it's best to figure them
out, then "cleaned them up."

>Are databases consistent when i press Ctrl-C running htdig?

Not necessarily. If you supply the -l flag to htdig, it will make
sure the databases are consistent and write out a list of the URLs it
still needs to index before exiting. Otherwise, the signal just kills
it in the middle and there's no guarantee it wasn't in the middle of
a write.


To unsubscribe from the htdig3-dev mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Wed Feb 23 2000 - 07:40:40 PST