Re: [htdig] Multiple Databases - How to?

Subject: Re: [htdig] Multiple Databases - How to?
From: Gilles Detillieux (
Date: Wed Aug 23 2000 - 08:13:51 PDT

According to Mike Potter:
> I have read the archives of the mailing list and attempted to get multiple
> databases working, but have been unsuccessful so far.
> Here is what I would like:
> I have 4 databases, each with its own .conf file and db directory. I would
> like the user to be able to search any of the four, or all of them (I call
> that database "all"). I have made a conf file named "all.conf" and a db
> directory called "all". In the all.conf file, it specifies the same
> information as one of the previous databases (the smallest one). How
> exactly do I merge the 4 databases into one? I think that I run htdig onf
> the all.conf file, and then merger the other 3 into that one?
> I am using htdig version 3.1.5.
> I appreciate all the help. If I can get this working, I'd be happy to
> write a tutorial or instructions on how to do it so that others can learn.

cp /opt/www/htdig/db/one/* /opt/www/htdig/db/all/
htmerge -c /opt/www/htdig/conf/all.conf -m /opt/www/htdig/conf/two.conf
htmerge -c /opt/www/htdig/conf/all.conf -m /opt/www/htdig/conf/three.conf
htmerge -c /opt/www/htdig/conf/all.conf -m /opt/www/htdig/conf/four.conf

This assumes that "one" and "all" are two distinct databases in
separate directories. This allows you to do updates on the 4 individual
smaller databases independently, and after any update you can repeat
the procedure above.

You could also do it the way you suggested, using the smallest database
in the "all" directory, but if you tried to update it after merging in
the other 3, htdig would check all documents in the merged database.

However, you could do an initial dig of the smaller database, assuming
the start_url in all.conf represents this smaller document set, and merge
in the other 3, as:

htdig -c /opt/www/htdig/conf/all.conf -i
htmerge -c /opt/www/htdig/conf/all.conf
htmerge -c /opt/www/htdig/conf/all.conf -m /opt/www/htdig/conf/two.conf
htmerge -c /opt/www/htdig/conf/all.conf -m /opt/www/htdig/conf/three.conf
htmerge -c /opt/www/htdig/conf/all.conf -m /opt/www/htdig/conf/four.conf

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Wed Aug 23 2000 - 08:14:30 PDT