Re: [htdig] serious htdig 3.1.5 problems


Subject: Re: [htdig] serious htdig 3.1.5 problems
From: Tim Rightnour (root@garbled.net)
Date: Tue Feb 29 2000 - 10:08:29 PST


On 29-Feb-00 Geoff Hutchison wrote:
>> (there are hundreds of these deleted lines)
>> Deleted, no excerpt:
>> 250503/http://mail-index.netbsd.org/tech-misc/2000/01/20/0003.html
>
> There are many reasons this can happen. The page can be forbidden by
> robots.txt files or noindex tags. Also, if the indexing stops before the
> pages are retrieved (e.g. server_max_docs) then these pages will be in the
> database but have not been indexed.

I will check the server_max_docs.. perhaps that is set too low.

>
>> DB2 problem...: /home/nbsdwww/htdig/db/db.docdb: page 99694 doesn't exist,
>> create flag not set
>
> These seem to come up periodically. AFAIK, we have not been able to get a
> reproducible test case for why they come up. If you can consistently
> reproduce this, it would be very helpful. For the bad news, it's a sign
> that the database is corrupted. :-(

Yes.. the database is definately corupt.. I can at least tell you that much.
And yes.. I can consistently repeat this.. I deleted the entire db dir, and ran
rundig again.. and got the exact same message.

>
> (There is, of course, no guarantee that there is only one cause for this
> sort of message. But since we haven't found *any* causes, even one would
> be good!)

I'd be glad to help.. because right now my search engine is hosed.. and I can't
index the site. This is a Bad Thing(tm). ;)

---
Tim Rightnour <root@garbled.net>
NetBSD: Free multi-architecture OS http://www.netbsd.org/
NetBSD Mailing lists on the web: http://mail-index.netbsd.org/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Tue Feb 29 2000 - 10:13:10 PST