Re: htdig: Different Databases for htdig

Michael Graff (
14 Sep 1998 14:41:10 -0700

Michael Glauche <> writes:

> Therefor I want to try to implement another database backend for htdig.
> (esp. for mysql ... I'ts one of the fastest I've seen)

Please make the database layer generic enough that I can hook it into
PostgreSQL without serious difficulty. I'll write the stubs; what I
mean is don't litter the htdig source with mysql_foo() calls, do it as
a generic database layer.

> - is it worth ? :)

I don't know if it is worth it or not, but I've considered writing
something like htdig, using PostgreSQL as the database server, but
I've not yet done this.

It seems that at the very least it will slow down digging, and perhaps
retrieval a bit, but should be far less of a memory pig. And, if the
database is done right, one could have multiple diggers running at a
time, dumping into the same database, to index a site in parallel.

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:47 PST