htdig: htdig limitations

Iosif Fettich (
Sat, 7 Feb 1998 12:06:10 +0200 (EET)


I'm new to the list, so I don't know if what I'm complaining about is
already solved someway or not; I had a problem with max_doc_size.

Actually, after a while, indexing lots of files showed no progress:
altough new files appeared, they wasn't indexed any more, as if the number
of indexable files was limited. Digging a bit through the sources and
looking on the logs, it became obvious that the bad number here was the
default max_doc_size set to 1000000. Increasing the number solved the
problem and we are operational now, but it's rather odd:

As I got it, max_doc_size should limit the size of _one_ document accepted
for indexing, not the number of files possible to use...

It looks to me that there is an intermediate, 'virtual' document, keeping
infos about the files that will be indexed. If this temporary document
increases in size, indexing will stop.

Is this a bug, did I miss the point and my solution is working by pure
hasard, or what?

Thanks for any reply.

Iosif Fettich

Iosif Fettich | e-mail: ICQ UIN: 5496730
Mng. Director | phone/fax: +40-(0)65-162614
NetSoft SRL | mail: NetSoft SRL,4300 Tg.Mures,O.P.1-C.P.172,Romania

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:25:41 PST