Re: [htdig] Multiple servers in one database

Marian Steinbach (
Mon, 07 Jun 1999 14:49:14 +0200

> The situation is:
> * i have got a large list (several thousands) of servers to index (i know that
> htdig is only for small intranets etc, but suppose i'm mad and i've got unlimited
> amount of disk space)
> * i want to have a searchable (with htsearch) database of them all -
> preferably in one file (i.e. i want to be able to search all of them with a
> single form query)
> therefore:
> the list is too long to include it in a single config file after start_url.
> (or maybe it isn't - will htdig be able to read several-hundred-kilobytes-long line)?
> so i have to make multiple config files and run htdigs serially.

As far as I know you can tell htdig to read the URL list from an
external text file. The list article and other replies in this
thread deal with that issue.

The documentation explains this very short in .

Hope your new Search Engine will be a real benefit for the web!!!

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Mon Jun 07 1999 - 05:03:17 PDT