Re: [htdig] large db's and RAM


Geoff Hutchison (ghutchis@wso.williams.edu)
Fri, 17 Sep 1999 09:09:58 -0500


At 2:40 AM -0600 9/17/99, Jim Cole wrote:
>have to pull a bunch of bits off of the drive every time it is invoked,
>regardless of how much memory you have. I am sure someone will be happy to
>correct me if I am wrong ;)

This is true, but only partially. Almost every OS around has a disk
cache to speed up disk accesses. Linux, and I'm sure others, expands
the cache to use whatever memory isn't being used at the moment. So
adding in a bunch of RAM will expand the cache and keep more things
in cache. Beyond that, large searches that might require paging won't
happen and that will also provide a speed boost.

On the other hand, I personally wouldn't just run out and buy a pile
of RAM even with the money. I'd add it a little bit at a time and see
how effective it was.

>The only option I can think of for keeping the databases in memory would be to
>look at setting up some sort of RAM disk and populating it with the databases
>whenever the system starts up. This is not something I have actually tried and
>I have no idea whether it would work or not.

This would almost assuredly work. I'd probably put the CGI on the RAM
disk as well. I won't pretend you'll get the same speedup as if there
were sophisticated caching schemes in the CGI, but it's definitely
going to help. (As for the cache schemes, well, contributions are
always welcome ;-)

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.



This archive was generated by hypermail 2.0b3 on Fri Sep 17 1999 - 07:19:24 PDT