Subject: RE: [htdig] suggestions for large multi-server indexing?
From: Harold Sinclair (email@example.com)
Date: Fri Jun 09 2000 - 10:40:14 PDT
On Fri, 9 Jun 2000, Budd, S wrote:
> We index using 3.1.3
> htmerge: Total documents: 177428
> htmerge: Total doc db size (in K): 1676650
> from 181 servers. It takes about 28 hours
> with one initial dig and another update dig
> two hours later.
How are you doing the dig -- each server individually with its own
configuration file and then merging all into one big database at
the end? Or somehow else..?
> -----Original Message-----
> From: Albert Lunde [mailto:Albert-Lunde@nwu.edu]
> Sent: Friday, June 09, 2000 5:11 PM
> To: firstname.lastname@example.org
> Subject: [htdig] suggestions for large multi-server indexing?
> (1) What are the are the pros and cons of doing a single big index
> (giving it starting URLs across all servers) vs. doing a number of
> small indexes and merging them?
I'd like to hear your comments on this!
To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.
This archive was generated by hypermail 2b28 : Fri Jun 09 2000 - 08:31:13 PDT