Re: [htdig] update dig problems

Subject: Re: [htdig] update dig problems
From: Gilles Detillieux (
Date: Fri Sep 01 2000 - 08:23:10 PDT

According to Adam Rice:
> I'm having a problem where my search results are out-of-date with
> respect to the site, even though htdig is definitely running, and
> definitely fetching the files from the web server, and not giving
> errors. Perhaps I am misunderstanding what an update dig does? I thought
> that it checked every document in its database, and rescanned it if it
> was new, as well as following any links to new documents, and removing
> it if it gets a 404.
> I run htdig and htmerge with the -a commandline options. I then move the
> *, * and * files to *.docdb,
> * and *.words.db respectively. I don't actually use
> wildcards, the *s are just there because I have different databases for
> different sites. I then copy the *.docdb file back to * so
> that it is there for the next update dig. The * file is
> left alone ready for the next update.
> Does that procedure sound correct? All the pages on the sites use
> server-side includes, and hence don't have Last-Modified: headers, could
> that be confusing matters?

The procedure above sounds correct to me, but for dynamic content with no
Last-Modifed headers, you need to set

 modification_time_is_now: true

in your configuration file for 3.1.x. In 3.2.0, this attribute is gone,
and htdig always assumes the current time for any missing Last-Modified

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Fri Sep 01 2000 - 08:24:34 PDT