Re: [htdig] Advice wanted: Multiple mailing lists

Subject: Re: [htdig] Advice wanted: Multiple mailing lists
From: Geoff Hutchison (
Date: Thu Mar 16 2000 - 19:28:20 PST

On Thu, 16 Mar 2000, David Gibbs wrote:

> My question is: What is the best way to build my search indexes? Should I
> have one large database with a search filter restriction, or should I have
> multiple databases (one for each mailing list archive)?

This depends considerably on how big these archives are going to be. On
the WSO site, we just have one big database (now around 80,000 URLs) with
mailing lists, student pages, etc. So anyone restricts with the search
form restrict and exclude fields.

I know of at least two mailing list archive sites that have multiple
databases. But these folks index hundreds of high volume (e.g.
linux-kernel and bugtraq) mailing lists.

So my suggestion depends on the volume you expect to receive. If you think
you might have multi-GB of data combined, you probably want to split them.
It also depends a bit on whether users might want to search all of them at

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Thu Mar 16 2000 - 18:25:37 PST