Questions for the master of ht://dig


default user (default@msstate.edu)
Fri, 29 May 1998 15:38:54 -0500


I've got some questions and I didn't find them in the maillist archives,
so I thought I'd send you mail.

First, this is the machine we setup for experimental use of ht://dig. A
486DX2/66 with 64 megs RAM, 6 gig IDE drive with 100 meg swap, running
RedHat Linux with kernel v2.0.33. Web server (as if it mattered :)) is
Apache v1.2.6. If I'm able to get reasonable results with ht://dig, it
will replace Harvest 1.5.17 on a PentiumPro 200 with 64 megs RAM running
Slackware Linux with kernel 2.0.33 and will be the main search engine
for the Mississippi State University website (www.msstate.edu).

1) Is 3.0.8b2 of ht://dig the most recent version? If not, where is it?

2) After digging about 25,000 documents on our site (with appox 50+ web
servers) ht://dig will drop core and quit. It will happen on different
pages and different servers with no pattern. This is less than half of
the indexable documents in our domain. Harvest's gatherer is capable of
indexing appox 73,000 in this same domain. (for a comparision, our
Harvest stats are at
http://msuinfo.ur.msstate.edu/Harvest/brokers/msstate/stats.html ). Is
there a way to stop the core dump?

3) Speaking of statistics, is there a utility that will report the stats
of the ht://dig database, primarily the number of pages pulled from each
webserver and report it in html form?

4) Even with 64 megs RAM and a swap partition of almost 100 meg, htmerge
is still giving the "Virtual memory exceeded in 'new'" error. Is there
any way to force htmerge to use only memory available?

Thank you for your attention in this.

Chris Brown
brownc@ur.msstate.edu
University Relations - New Media Division
Mississippi State University



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:19 PST