Re: [htdig] Indexing huge websites (really Linux 2G max file size


Geoff Hutchison (ghutchis@wso.williams.edu)
Thu, 05 Aug 1999 09:06:14 -0400


loic@ceic.com wrote:
>
> marc@nozell.com writes:
> >
> > I maintain a very large archive (archiver.rootsweb.com) of almost
> > 9,000 mailing lists with more being added every day. Currently we
> > have 900,000 messages (7.5G) which is also growing rapidly.
>
> Transparent compression might be a solution :
> There also is bigfile support for linux, glibc-2.1 required. I don't

The transparent compression patch is:
http://linux-patches.rock-projects.com/v2.2-f/e2compr.html

The large filesystem patch is:
http://linux-patches.rock-projects.com/v2.2-f/lfs-32bit.html

The latter is also usually rolled into the -ac patches to the Linux
kernel.

I don't know how well tested they are, but I imagine for ext2 on ia32,
they're probably pretty heavily tested. That doesn't mean they are
highly supported patches. ;-)

-- 
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig@htdig.org containing the single word unsubscribe in the SUBJECT of the message.



This archive was generated by hypermail 2.0b3 on Thu Aug 05 1999 - 06:07:12 PDT