[htdig] Disk Space

Tim Perdue, Geocrawler.com (tim@geocrawler.com)
Sun, 23 May 1999 16:05:05 -0500

I've succeeded in filling up the shiny new 18GB disk I bought for ht://dig,
and I am wondering if you have some further recommendations for reducing
disk usage.

I read the FAQ and also set up a process to gzip the db.wordlist. I'm using
the process you recommended for my "update digs" (digging the new links
only, and then merging those with the old database with the -m option). It
appears the htmerge -m option does require the old db.wordlist to be

Here is one example of the 500 databases I have now:

-rw-r--r-- 1 root root 51584000 May 23 14:25 db.docdb
-rw-r--r-- 1 root root 2388992 May 23 14:25 db.docs.index
-rw-r--r-- 1 root root 12838976 May 23 14:22 db.wordlist.gz
-rw-r--r-- 1 root root 46101504 May 23 14:22 db.words.db
-rw-r--r-- 1 tim users 90 May 23 14:16 dig.log
-rw-r--r-- 1 tim users 106 May 23 14:25 merge.log

Any more suggestions would be appreciated. This software really is working
great and the performance is pretty impressive too!

Tim Perdue
PHPBuilder.com / GotoCity.com / Geocrawler.com

To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Sun May 23 1999 - 13:31:34 PDT