Re: [htdig] Indexing huge websites (really Linux 2G max file size

Geoff Hutchison (
Thu, 05 Aug 1999 09:06:14 -0400 wrote:
> writes:
> >
> > I maintain a very large archive ( of almost
> > 9,000 mailing lists with more being added every day. Currently we
> > have 900,000 messages (7.5G) which is also growing rapidly.
> Transparent compression might be a solution :
> There also is bigfile support for linux, glibc-2.1 required. I don't

The transparent compression patch is:

The large filesystem patch is:

The latter is also usually rolled into the -ac patches to the Linux

I don't know how well tested they are, but I imagine for ext2 on ia32,
they're probably pretty heavily tested. That doesn't mean they are
highly supported patches. ;-)

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to containing the single word unsubscribe in the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Thu Aug 05 1999 - 06:07:12 PDT