[htdig] Re: 3.1.5 strange freeze problem


Subject: [htdig] Re: 3.1.5 strange freeze problem
From: Peter L. Peres (plp@actcom.co.il)
Date: Sun Oct 08 2000 - 15:11:13 PDT


Hello,

I run 3.1.5 with my own little patch (see htdig archives for
prune-parent-dir patch) on a Linux system (Suse base, modified). Recently
I have run into a snag: htdig stops at some point in the search (after a
few hours - this is a 233 MHz K6 machine doing nothing else).

lsof on the htdig process showed no files open but for the terminal, the
databases and a few other things (no files were being read in).

gdb attached to the running process yielded this stack trace:

Reading symbols from /usr/lib/libz.so.1...done.
Reading symbols from /usr/lib/libstdc++.so.2.9...done.
Reading symbols from /lib/libm.so.6...done.
Reading symbols from /lib/libc.so.6...done.
Reading symbols from /lib/ld-linux.so.2...done.
Reading symbols from /lib/libnss_files.so.1...done.
0x400c86b0 in chunk_free (ar_ptr=0x40122300, p=0xa04af30) at malloc.c:2969
malloc.c:2969: No such file or directory.
(gdb) bt
#0 0x400c86b0 in chunk_free (ar_ptr=0x40122300, p=0xa04af30) at
malloc.c:2969
#1 0x400c83c1 in __libc_free (mem=0xa04af38) at malloc.c:2872
#2 0x4005b2f2 in __builtin_vec_delete (ptr=0xa04af38)
#3 0x805ea34 in _start ()
#4 0x805ec66 in _start ()
#5 0x805ca82 in _start ()
#6 0x8050f8a in _start ()
#7 0x80508a2 in _start ()
#8 0x8055cbc in _start ()
(gdb) c
Continuing.

Program received signal SIGINT, Interrupt.
0x400c86b0 in chunk_free (ar_ptr=0x40122300, p=0x94e9e50) at malloc.c:2969
malloc.c:2969: No such file or directory.
(gdb)
bt
#0 0x400c86b0 in chunk_free (ar_ptr=0x40122300, p=0x94e9e50) at
malloc.c:2969
#1 0x400c83c1 in __libc_free (mem=0x94e9e58) at malloc.c:2872
#2 0x4005b2f2 in __builtin_vec_delete (ptr=0x94e9e58)
#3 0x80611d0 in _start ()
#4 0x809748c in _start ()
#5 0x805ea48 in _start ()
#6 0x805ea5a in _start ()
#7 0x805ea5a in _start ()
#8 0x805ea5a in _start ()
#9 0x805ec66 in _start ()
#10 0x805ca82 in _start ()
#11 0x8050f8a in _start ()
#12 0x80508a2 in _start ()
#13 0x8055cbc in _start ()
(gdb)

(Note SIGINT and then again stack trace)

The htdig seems to be turning in little circles at this point. Any help in
fixing this problem would be much appreciated. Please CC: plp@actcom.co.il
as I'm not currently on the htdig list(s) (but I will be asap).

It seems to be some sort of broken C++ library problem, therefore here is
ldd output on htdig (can you see anything wrong here ?):

plp:/usr/src/htdig-3.1.5 # ldd /opt/www/htdig/bin/htdig
        libz.so.1 => /usr/lib/libz.so.1 (0x4000c000)
        libstdc++.so.2.9 => /usr/lib/libstdc++.so.2.9 (0x4001c000)
        libm.so.6 => /lib/libm.so.6 (0x40060000)
        libc.so.6 => /lib/libc.so.6 (0x4007a000)
        /lib/ld-linux.so.2 => /lib/ld-linux.so.2 (0x2aaaa000)

ls -l on libs:

lrwxrwxrwx 1 root root 13 Jul 22 16:23 /usr/lib/libz.so.1 \
  -> libz.so.1.1.3
-rwxr-xr-x 1 root root 93836 Apr 15 1999 \
  /usr/lib/libz.so.1.1.3

lrwxrwxrwx 1 root root 18 Jul 22 16:23 \
  /usr/lib/libstdc++.so.2.9 -> libstdc++.so.2.9.0
-r-xr-xr-x 1 root root 1164502 Apr 15 1999 \
  /usr/lib/libstdc++.so.2.9.0

-rwxr-xr-x 1 root root 384589 Apr 15 1999 /lib/libm.so.6

-rwxr-xr-x 1 root root 2475225 Apr 15 1999 /lib/libc.so.6

lrwxrwxrwx 1 root root 11 Jul 22 16:51 /lib/ld-linux.so.2 \
  -> ld-2.0.7.so
-rwxr-xr-x 1 root root 148968 Apr 15 1999 /lib/ld-2.0.7.so

Last, I'd like to mention that the words db was about 270 MB at the time
of the fault, that I index a LOT of Apache generated directory indexes,
and that the problem does not happen if/when I remove large chunks of the
tree to be indexed from htdig's input (but the chunks are too large for me
to find what went wrong). Did I hit a system limit here ?

Note that I do not have an unpacked source for malloc.c (lib function)
thus I can't look easily. Maybe someone can ?

thank you very much in advance,

Peter

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Sun Oct 08 2000 - 14:13:37 PDT