Re: htdig: adding additional sites to index


Geoff Hutchison (ghutchis@wso.williams.edu)
Wed, 16 Dec 1998 14:09:48 -0500 (EST)


On Wed, 16 Dec 1998, Charlie Romero wrote:

> and then I want to add
>
> www.nextcompany4.com
> www.nextcompany5.com...
>
> how should I approach this?
>
> I don't want to have to reindex everything everytime I add a server to the
> list of sites being indexed.

You don't have to. If you do an update dig, it will only add the pages
that aren't in the databse already.

To do an update dig, don't specify "-i" to htdig. If you use "-a" you'll
need to have a copy of your db.wordlist and db.docdb as .work files as
well. After running htdig and htmerege, you'll have the new pages in there
and you won't have reindexed everything.

I do this by default in my digging script, and it usually cuts the
indexing time from 2 hrs to 20 min. Your performance will vary. :-)

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:29:52 PST