Re: [htdig] indexing over and over again

Subject: Re: [htdig] indexing over and over again
From: Geoff Hutchison (
Date: Thu Jun 08 2000 - 20:18:33 PDT

At 11:37 PM -0400 6/8/00, Clint Gilders wrote:
>verbose mode without the -i switch and point it to a new URL I notice
>that it still retrieves the previously indexed pages from other URLs
>but doesn't index them.

Right, without the -i flag, it's in "update mode." So it checks the
dates on all the URLs in the database to see if they've changed. If
they have, it reindexes them. If not, it goes to the next URL.

> Should I have it use a different database while digging these new
>sites, and then merge it with the original? If so, how would I go
>about configuring it (i'm still using the pre-fab rundig and

You can create a new config file (say new.conf) and use that for
indexing your new sites. Then you'd do something like this:

htmerge -m new.conf -c htdig.conf
htmerge -c htdig.conf

The first command will merge the databases specified by new.conf into
the others. The second command updates the databases for searching.

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Thu Jun 08 2000 - 18:09:33 PDT