[htdig] Large DB

Subject: [htdig] Large DB
From: Premier Hosting Administrator (sysadmin@premier-hosting.com)
Date: Thu Nov 04 1999 - 09:21:51 PST

We have been playing around with HtDig, UDMSearch, Catalog, and others
recently to find out which ones can handle large volumes of searches and
the db itself...

We are looking at indexing about 300,000 websites from top to bottom for
a specialized search engine and are concerned about performance..

If the DB could fit into RAM, we'd guess that performance may not be an
issue... but without buying 1-2 Gig of RAM at a major cost we'd rather
avoid that...

The machine for indexing will be a small PII server but the machine the
public searches on will be a PIII-500 with 128 meg of RAM...

Any benchmarks for such a size of DB etc.? Any information would be
greatly appreciated..


To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.

This archive was generated by hypermail 2b25 : Thu Nov 04 1999 - 09:36:15 PST