RE: [htdig] One solution for slow dig on Linux.

Subject: RE: [htdig] One solution for slow dig on Linux.
From: Sean Pecor (
Date: Tue Dec 21 1999 - 06:38:54 PST


> Our of curiousity, where does it say this? I see:
> 1.2. Can I index the internet with ht://Dig?

I stand corrected! It doesn't, literally, say it is incapable of doing so. I
simply took it as such during my initial hunt for a search indexing tool;
many of the alternatives wax poetic about how powerful their particular
engine is and lean clear from admitting any particular design limitation,
whatever its form.

> (BTW, we have lots of people indexing well above 150,000 URLs.)

Good! This particular project will bloat up to the near million page mark,
on a hundred thousand or so external servers, on pages that will need to be
re-indexed on a weekly basis (some on a daily basis). You've done a
fantastic job with ht://Dig!

All the best,


# Digital Spinner, Inc.
# Web Design, Development and Consulting.
# Phone: 802.948.2020
# Fax: 802.948.2749

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Tue Dec 21 1999 - 06:57:44 PST