Re: [htdig] Indexing a whole Intranet


Subject: Re: [htdig] Indexing a whole Intranet
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Fri Jan 07 2000 - 11:06:27 PST


At 2:35 PM +0100 1/7/00, Paul COURBIS wrote:
>Does everyone use htdig to index a whole Intranet (about 1000+ servers) ?

Probably not "everyone," but this is fairly common.

>Any advices on that subject ? How should I run htdig to be able to
>update database & to add new servers when requested in an easy way ?

If you're indexing all of a domain, say courbis.com (your e-mail
domain), you could try something like this:

start_url: http://www.courbis.com/
limit_urls_to: courbis.com

You can add any servers you want to start_url and it will "find" new
servers linked from the others as long as they're in the main domain.

As for updating the database, it does this by default when you re-run
htdig/htmerge. If you want a particular script that does this, see
any of a variety of examples at
<http://www.htdig.org/files/contrib/scripts/> (I'm a bit partial to
rundig.sh ;-)

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Fri Jan 07 2000 - 11:24:57 PST