[htdig] delayed remove of bad urls

Subject: [htdig] delayed remove of bad urls
From: st002556@hrz1.hrz.tu-darmstadt.de
Date: Tue May 23 2000 - 11:48:03 PDT

  I'm using htdig for a www-searchengine on a special topic. I start
htdig each week per cron to update the database. Now it seems as
sometimes there is a problem with connecting other servers. In this
cases htmerge removes all docs from the database! :o(

Is there a way to avoid this? I can set remove_bad_urls to false, but
in this case, bad url will never be removed! What I need is way
to tell htmerge somethink like:

Only remove bad urls, if they were last two weeks too!

How can I do something like this?


LARP-Welt, das LARP-Portal: http://larp-welt.de

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Tue May 23 2000 - 09:37:02 PDT