Re: [htdig] remove_bad_urls


Torsten Neuer (tneuer@inwise.de)
Tue, 26 Oct 1999 08:25:47 +0200


Maria Cervantes wrote:
>
> Hi, I'm using htdig-3.1.2 and I have to index 66 servers, but sometimes
> some of them can't be reached because they are down, etc. So I never get
> the whole list of servers indexed.
> However, I wish to keep in the database of the htdig, the documents of
> the servers that coudn't be reach, but were indexed in a other time.
> I put in the config file remove_bad_urls: false
> I run htdig three days ago and I got:
> htdig: Run complete
> htdig: 66 servers seen:
> htdig: iibce.edu.uy:80 162 documents
>
> I run htdig yesterday and I got:
> htdig: Run complete
> htdig: 65 servers seen:
> htdig: iibce.edu.uy:80 0 documents
>
> I searched for words I know are in this url and I didn't get any
> result.
> Do you have any idea??
> Did I misunderstand the usage of remove_bad_urls??

Do you "update" using an initial dig ("-i" command line option)?
In this case "remove_bad_urls" won't come into effect.

hth,
  Torsten

-- 
InWise - Wirtschaftlich-Wissenschaftlicher Internet Service GmbH
Waldhofstraße 14                            Tel: +49-4101-403605
D-25474 Ellerbek                            Fax: +49-4101-403606
E-Mail: info@inwise.de            Internet: http://www.inwise.de

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig@htdig.org containing the single word unsubscribe in the SUBJECT of the message.



This archive was generated by hypermail 2.0b3 on Mon Oct 25 1999 - 23:35:06 PDT