Re: [htdig] BAD TAG IN SERIALIZED DATA: 110 and DB2 error messages


Subject: Re: [htdig] BAD TAG IN SERIALIZED DATA: 110 and DB2 error messages
From: Gilles Detillieux (grdetil@scrc.umanitoba.ca)
Date: Thu May 11 2000 - 12:16:40 PDT


According to Alain FORCIOLI:
> We're running HTDIG 3.1.5 on a ix86 Redhat 6.1.
> Htdig currently indexes about 50000 documents (HTML and PDF)
> (http://www.unesco.org/).
>
>
> A Cron job make an incremental indexation every day (from Monday to
> Saturday). Sunday is a special day where an initial indexation is done.
>
> The following gives the situation after the initial (or incremental)
> indexation.
>
> # ls -al /usr/local/htdig/db
> total 2948548
> drwxr-xr-x 2 root root 110592 May 7 03:21 .
> drwxr-xr-x 6 root root 4096 May 3 17:52 ..
> -rw-r--r-- 1 root root 480542720 May 7 03:23 db.docdb
> -rw-r--r-- 1 root root 480542720 May 7 03:21 db.docdb.work
> -rw-r--r-- 1 root root 263168 May 7 03:23 db.docs.index
> -rw-r--r-- 1 root root 263168 May 7 03:21 db.docs.index.work
> -rw-r--r-- 1 root root 568581620 May 7 03:24 db.wordlist
> -rw-r--r-- 1 root root 568581620 May 7 03:21 db.wordlist.work
> -rw-r--r-- 1 root root 458707968 May 7 03:25 db.words.db
> -rw-r--r-- 1 root root 458707968 May 7 03:21 db.words.db.work
>
> Incremental and initial indexation frequently reports the following
> problems :
> ...
> BAD TAG IN SERIALIZED DATA: 110
> ...
> DB2 problem...: /usr/local/htdig/db/db.docdb.work: page 1191531215
> doesn't exist, create flag not set
> ...
>
> As a consequence, the search doesn't work properly.
> To solve this problem i have to reindex from scratch (i.e. initial
> indexation).
>
>
> I had a look at Htdig discussion list but unfortunately though this
> problem is clearly reported i don't know how to solve definitively
> this problem.

This seems like that elusive database corruption problem that pops up
from time to time. We haven't been able to get a simple, reproduceable
test case that would allow us to hunt it down and fix it. I'm afraid
this is a little out of my league, but perhaps you could work with Geoff
or Loic in an attempt to get to the bottom of this.

If you have no success in finding it, maybe you can help them gets some
bugs out of the 3.2 beta code instead. 3.2 uses a newer Berkeley DB
release, so it may have solved some problems that still exist in the 3.1.x
code, but of course there's a lot of new code that needs to be ironed out.

-- 
Gilles R. Detillieux              E-mail: <grdetil@scrc.umanitoba.ca>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/~grdetil
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Thu May 11 2000 - 10:04:58 PDT