Re: htdig: htmerge: Unable to open word list file....

J. op den Brouw (
Mon, 07 Dec 1998 12:39:11 +0100

This is what I needed to know. Thanks.

Gilles Detillieux wrote:
> According to J. op den Brouw:
> > I got the "htmerge: Unable to open word list file" error
> > and to my opinion it showed when I hit a server that uses
> > a robots.txt file and I could not get a file, then somewhere
> > htmerge crashes because nothing is indexed.
> >
> > I only wanted to index that site, no other.
> Yup, if htdig doesn't index anything at all, htmerge will give that error.
> If you're only indexing that one site, and their robots.txt file blocks
> out htdig (or any robots), then htdig will get 0 documents.
> You should contact the administrators of the site you want to index (a
> good courtesy to follow in any case), and ask them if they'll allow you
> to index their site. If they allow access to the User-agent "htdig"
> in their robots.txt file, you'll be able to index the directories to
> which they give htdig access.
> If they want to allow you, but not just any old htdig client, you can
> agree on some other user agent ID that they'll put in their robots.txt
> file, for you to use. You can configure this ID in your htdig.conf file,
> with the "user_agent" parameter.
> --
> Gilles R. Detillieux E-mail: <>
> Spinal Cord Research Centre WWW:
> Dept. Physiology, U. of Manitoba Phone: (204)789-3766
> Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930
To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:29:48 PST