Re: [htdig] indexing multiple web sites

Subject: Re: [htdig] indexing multiple web sites
From: Geoff Hutchison (
Date: Wed May 03 2000 - 06:28:37 PDT

At 6:09 AM -0500 5/3/00, atta dubson wrote:
>i want to index several web sites and have them stored in one db.* set of
>files, yet i would like to at times reindex only certain sites and update
>only that part of the database (or add an additional site to an existing
>database) without having to reindex all the sites. what is the best way
>to go about that (htdig 3.1.5)?

As they would say in New England "yah can't get there from here."

In 3.1.5 if you want one database, you'll have to accept that updates
will occur on all URLs.

However, you could have separate databases for each group you want to
update and then use htmerge -m to merge them two-at-a-time until you
have the one database for searching.

In 3.2 there are a few ways of doing this, by specifying the list of
URLs through standard input, or by using -m and a file of URLs
specifying the exact list of URLs to update/index. (The latter was
broken until this last snapshot.)

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Wed May 03 2000 - 04:29:08 PDT