htdig: htmerge using 4GB


Geoff Hutchison (Geoffrey.R.Hutchison@williams.edu)
Fri, 22 May 1998 11:44:22 -0400


Hello again,

I remember a thread a while back about htmerge using lots of disk space.
Well I started running out of space for sorting a while ago, moved from one
drive to another until last night it outstripped a 4GB drive.

I'm using GNU sort and I set TMPDIR to move it to the 4GB drive. The
problem simply seems to be (as seen from the cron output) that it's using
more than 4GB of temporary files!

>Start time: Fri May 22 05:00:00 EDT 1998
>Done Digging: Fri May 22 06:23:01 EDT 1998
>/usr/local/bin/sort: write error: No space left on device
>Done Merge Step 1
>Done Merge Step 2
>Done Merging: Fri May 22 06:57:44 EDT 1998
>End time: Fri May 22 06:57:44 EDT 1998

But I can't believe this, my databases are "only" 100-200MB each:
-rw-r--r-- 1 root other 183165924 May 22 06:57 db.docdb
-rw-rw-r-- 1 root other 9249878 May 22 06:57 db.docs.index
-rw-r--r-- 1 root other 152509072 May 22 06:22 db.wordlist
-rw-rw-r-- 1 root other 103286268 May 21 22:39 db.words.gdbm

Can htmerge really be using about 40x the size of the database to sort?

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:17 PST