Re: [htdig] htdig missing subdirectories (was: Incremental indexing)

Subject: Re: [htdig] htdig missing subdirectories (was: Incremental indexing)
From: Gilles Detillieux (
Date: Wed Dec 13 2000 - 10:10:34 PST

Please direct your questions to the list, not to me personally.
See FAQ 1.16. Also, you're off topic, as this has nothing to do with
last week's "Incremental indexing" thread, so you should pick a more
descriptive subject.

According to crosstar:
> I have copiously poured over the messages
> in the mailing list, as well as references in FAQ.
> I am not very technical, but my situation is that htdig is
> missing a lot of files, words and subdirectories, altogether.
> I'm wondering if there is a simpler adjustment in
> htdig.conf to remedy this? I simply do not understand
> the instrtuctions, as given, unfortunately, and note that
> one reader says that he thinks tinkering with the
> server is not the answer.

Did you follow the recommendations in FAQ 5.25 & 5.27? That's probably
where you should focus your attention. Running htdig with the -vvv
option will give you tons of output, but if you trace your way through
there you might be able to see why it's missing parts of your site.

> I tried running htfuzzy but get the error:
> htfuzzy: No algorithms specified

You need to tell htfuzzy which database to build. This won't solve your
problem above, though. It's just for building databases for fuzzy match

> I have changed one default up upping to:
> max_head_length: 50000

That will make htdig keep more of each document for use in excerpts for
matched pages, but it won't get you more matches. However, upping the
max_doc_size may get htdig to index more stuff if it was missing links from
really large pages. See FAQ 5.1.

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Wed Dec 13 2000 - 10:20:30 PST