Re: [htdig] Indexing a whole Intranet

Subject: Re: [htdig] Indexing a whole Intranet
From: Geoff Hutchison (
Date: Fri Jan 07 2000 - 11:06:27 PST

At 2:35 PM +0100 1/7/00, Paul COURBIS wrote:
>Does everyone use htdig to index a whole Intranet (about 1000+ servers) ?

Probably not "everyone," but this is fairly common.

>Any advices on that subject ? How should I run htdig to be able to
>update database & to add new servers when requested in an easy way ?

If you're indexing all of a domain, say (your e-mail
domain), you could try something like this:


You can add any servers you want to start_url and it will "find" new
servers linked from the others as long as they're in the main domain.

As for updating the database, it does this by default when you re-run
htdig/htmerge. If you want a particular script that does this, see
any of a variety of examples at
<> (I'm a bit partial to ;-)

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Fri Jan 07 2000 - 11:24:57 PST