Re: [htdig3-dev] Segmentation fault in long run

Subject: Re: [htdig3-dev] Segmentation fault in long run
Date: Tue Feb 22 2000 - 06:03:43 PST

> hmm, allmost all sites i crawl are external ones. i am not sure but i
> think that it doesnt depend on content of those pages, just thereis some
> critical point where it fails.


> Maybe you can try to index more that 10000 not so small documents
> localy, i.e. with your own documents first using my config, of course if
> you have so many. If you still want my file of urls, let me know, and
> i'll send them.

 I doubt very much that it's a matter of volume. We've indexed 2 millions
html documents using htword (not with htdig, though) without problems. Could
you send your config file ? I'll start crawling and wait for it to crash
(hopefully :-).


		Loic Dachary

24 av Secretan 75019 Paris Tel: 33 1 42 45 09 16 e-mail: URL:

------------------------------------ To unsubscribe from the htdig3-dev mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Tue Feb 22 2000 - 04:45:41 PST