Re: [htdig] Database Update

Subject: Re: [htdig] Database Update
From: Chad Phillips (
Date: Mon Aug 07 2000 - 08:25:39 PDT

I tried the -a option but still no luck. It uses alternate files, so I can still use the search engine while htdig is indexing. But when htdig/htmerge is finished, my old database is wiped out so I loose all the data from prior searches.

I'm sure I am missing some option somewhere. Do most people re-index from scratch?

>>> "Marcel Hicking" <> 08/06/00 12:50PM >>>
Run both htdig and htmerge with option -a
(= use alternate work file)
htdig/merge will then work on the file called s
instead of the current .db files.
See htdig --help ;-)

Then copy the workfiles over the actual database.

( cd $DBDIR && test -f &&
for f in *.work
    # mv -f $f `basename $f .work`
    # Keep alternate work files for next upgrade dig
    cp -f $f `basename $f .work`
done )

See the file "rundig" in the ht://dig bin dir for details.
Somewhere around line 40.

But as far as I remeber, htdig always does an update dig
unless you use -i
Try htdig -vv / htmerge -vv to see what is actually

Depending on the site you are digging, the following
might be helpful as well:
I have a server where all content is generated "on the fly"
from a database. So the document date is always "now"
causing htdig to retrieve the full site on every run.
No very nice since this is a big site, and I can't run the
dig via local filesystem naturally.
But this is actually not a ht://dig problem but lies with
the site's content. (And I haven't found a workaround
yet since the date of last document change is not even
in the database ;-( )

Cheers, Marcel

On 4 Aug 00, at 15:13, Chad Phillips wrote:

> How can I do an update on the Htdig database, with out deleting what is there? I had thought that the original database would kept as long as htdig was not run with the -i flag.
> The database I have took a long time to build and I want to add one more site, but I don't want to start the indexing from scratch. Should I build another database for the one site and then run htmerge?
> thanks
> chad

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Sun Aug 06 2000 - 22:25:24 PDT