Re: htdig: Using htdig just a few hours per day

Geoff Hutchison (
Tue, 1 Dec 1998 21:19:39 -0500

At 10:14 AM -0500 12/1/98, Gianugo Rabellino wrote:
>per site), so I understand that size shouldn't be a big
>issue. Should the number of sites be a problem, I might
>reduce it to the most important 50-100 of them.

Not really a problem. The problems with size have more to do with the
number of pages and words and the amount of each page stored.

>Is there a way to do that? I understand that the "-a"
>option might be helpful, since it keeps a copy of the
>existing database, but I don't see how I can tell
>htdig to "resume" a suspended run (and even how to
>suspend it: I don't know if htdig would behave if I
>send him SIGSTOPs and SIGCONTs via cron).

Well you can always try it. To save the most time on htdig, make sure you
don't use "-i" and that there's an old copy of the database around (e.g.
with "-a" make sure there are .work files from previous runs).

This will also get easier when I actually sit down and write the code to
merge multiple databases. With the semester drawing to a close, I don't
think that's going to happen for a few weeks. If it makes you feel any
better, it's the top *feature* that's on my list.

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:29:44 PST