Re: [htdig] Database Update

Subject: Re: [htdig] Database Update
From: David Adams (
Date: Tue Aug 08 2000 - 02:25:39 PDT

On Mon, 07 Aug 2000 10:25:39 -0500 Chad Phillips <>

> I tried the -a option but still no luck. It uses alternate files,
so I can still use the search engine while htdig is indexing.
But when htdig/htmerge is finished, my old database is wiped out so
I loose all the data from prior searches.
> I'm sure I am missing some option somewhere. Do most people re-index from scratch?

I'm having trouble understanding your question. What is this "data
from prior searches" that is being lost? Why do you want both to
update your database and keep it unchanged?

I re-build our major index from scratch once a week and update it
daily. A second index is simply re-built once a week. It all works
very smoothly.

> >>> "Marcel Hicking" <> 08/06/00 12:50PM >>>
> Run both htdig and htmerge with option -a
> (= use alternate work file)
> htdig/merge will then work on the file called s
> instead of the current .db files.
> See htdig --help ;-)
> Then copy the workfiles over the actual database.
> ie:
> ( cd $DBDIR && test -f &&
> for f in *.work
> do
> # mv -f $f `basename $f .work`
> # Keep alternate work files for next upgrade dig
> cp -f $f `basename $f .work`
> done )
> See the file "rundig" in the ht://dig bin dir for details.
> Somewhere around line 40.
> But as far as I remeber, htdig always does an update dig
> unless you use -i
> Try htdig -vv / htmerge -vv to see what is actually
> happening.
> Depending on the site you are digging, the following
> might be helpful as well:
> I have a server where all content is generated "on the fly"
> from a database. So the document date is always "now"
> causing htdig to retrieve the full site on every run.
> No very nice since this is a big site, and I can't run the
> dig via local filesystem naturally.
> But this is actually not a ht://dig problem but lies with
> the site's content. (And I haven't found a workaround
> yet since the date of last document change is not even
> in the database ;-( )
> Cheers, Marcel
> On 4 Aug 00, at 15:13, Chad Phillips wrote:
> > How can I do an update on the Htdig database, with out deleting what is there? I had thought that the original database would kept as long as htdig was not run with the -i flag.
> >
> > The database I have took a long time to build and I want to add one more site, but I don't want to start the indexing from scratch. Should I build another database for the one site and then run htmerge?
> >
> > thanks
> > chad

David Adams

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Mon Aug 07 2000 - 16:25:26 PDT