Geoff Hutchison (firstname.lastname@example.org)
Thu, 13 May 1999 11:28:58 -0400 (EDT)
On Thu, 13 May 1999, Brandon LaBonte wrote:
> We think about 250 servers, and probably 100,000 unqiue URL's....Does this
> exceed the reasonable capacity of htdig?
No, but it will take some time for an initial index. Each server slows
things down a bit with the added connections.
> Also, we are running this on a P2 with only 64 megs of Ram, and IDE Drives.
> We are trying to get the ram to 128 (maybe 256), and the drives to SCSI.
> Will this make a marked improvement in performance, or is this more an
> application issue?
You will almost definitely need more RAM. You can get by with virtual
memory, but you're going to be swapping your data a lot. I index around
70,000-75,000 URLs from one server using a P2 with 128MB RAM and a SCSI
drive. I get fairly good performance, but obviously updating the database
takes much less time than the first indexing.
Williams Students Online
To unsubscribe from the htdig mailing list, send a message to
email@example.com containing the single word "unsubscribe" in
the SUBJECT of the message.
This archive was generated by hypermail 2.0b3 on Thu May 13 1999 - 08:39:00 PDT