Re: [htdig] Re: 3.1.5 strange freeze problem (fwd)


Subject: Re: [htdig] Re: 3.1.5 strange freeze problem (fwd)
From: Andrew Scherpbier (ascherpbier@evoke.com)
Date: Mon Oct 09 2000 - 16:34:05 PDT


"Peter L. Peres" wrote:
>
> >swapping
>
> No, it was not swapping much just then but it is swapping a LOT when it
> runs. I have 32 MB RAM and 150 MB swap (on several disks). The swap space
> gets used up to 80-90 MB at times (when running X11 concurrently, with
> surprising performance under the circumstances, and load >2.4 ;-). Of
> course I was not doing that at the time.

Does the htdig process get very big?

> No, the machine seemed to be somewhat frozen. It sat there apprently doing
> nothing for hours (overnight) with load at 1.5 or higher. I think it's a
> bug. When I reduce the size of the input then I get completion in ~3
> hours. I expect the full shebang to complete in max. 8 hours, but this
> program has been run for 14 hours and did not finish. The symptoms are
> always the same.
>
> Would there be some way to trace the program more closely and see what it
> does ? Any pointers ? The code seems to be part of the C++ class
> destructor but I am not sure.
>
> PS: I'm on the list now, no need to CC.
>
> Peter

It would probably help to run htdig with a single '-v' option so that you can
see what page it is working on when it stops. (Make sure you either redirect
the output or run htdig from within 'script' because there will be a single
line per web page and it sounds like you have a lot of pages...)

If all else fails, you could also run indexes on smaller chunks of your tree
and merge them together to get the full index.

-- 
Andrew Scherpbier, Chief Scientist
Evoke Communications (http://www.evoke.com/)
8334 Clairemont Mesa Blvd #204, San Diego, CA 92111
phone +1.858.278.2329 x115   fax +1.858.278.2502

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Mon Oct 09 2000 - 16:38:22 PDT