[htdig] Speed of large search engine sites?

Charlie Romero (charlie@jumpinternet.com)
Sat, 11 Sep 1999 14:35:58 -0400

So how do they do it?

How can the major search engines retrieve data so fast? As quick as
some search results are they have got to be cached in RAM. How do
they cache a three terrabyte database in RAM? or at least fool me
into thinking it is.

I mean UW fast SCSI is great and so is a 200MHZ bus speed and so is a
Gig of RAM... How can I make htdig that fast if my .db's won't all
fit in RAM?


To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Sat Sep 11 1999 - 11:39:44 PDT