Re: htdig: htmerge using 4GB


ing. J.E.J. op den Brouw (jesse@crytonII.st.hhs.nl)
Sat, 23 May 1998 15:50:22 +0200 (MET DST)


On Fri, 22 May 1998, Geoff Hutchison wrote:

> Hello again,
>
> I remember a thread a while back about htmerge using lots of disk space.
> Well I started running out of space for sorting a while ago, moved from one
> drive to another until last night it outstripped a 4GB drive.
>
> I'm using GNU sort and I set TMPDIR to move it to the 4GB drive. The
> problem simply seems to be (as seen from the cron output) that it's using
> more than 4GB of temporary files!

Did you export the environment variable. I was fooled once by this.

TMPDIR=/whatever/your/path/is
export TMPDIR

... the rest here

> >Start time: Fri May 22 05:00:00 EDT 1998
> >Done Digging: Fri May 22 06:23:01 EDT 1998
> >/usr/local/bin/sort: write error: No space left on device
> >Done Merge Step 1
> >Done Merge Step 2
> >Done Merging: Fri May 22 06:57:44 EDT 1998
> >End time: Fri May 22 06:57:44 EDT 1998
>
> But I can't believe this, my databases are "only" 100-200MB each:
> -rw-r--r-- 1 root other 183165924 May 22 06:57 db.docdb
> -rw-rw-r-- 1 root other 9249878 May 22 06:57 db.docs.index
> -rw-r--r-- 1 root other 152509072 May 22 06:22 db.wordlist
> -rw-rw-r-- 1 root other 103286268 May 21 22:39 db.words.gdbm
>
> Can htmerge really be using about 40x the size of the database to sort?
>
> -Geoff Hutchison
> Williams Students Online
> http://wso.williams.edu/
>
>
> ----------------------------------------------------------------------
> To unsubscribe from the htdig mailing list, send a message to
> htdig-request@sdsu.edu containing the single word "unsubscribe" in
> the body of the message.
>

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:17 PST