Re: [htdig] htdig - spider or browser?

Subject: Re: [htdig] htdig - spider or browser?
From: Geoff Hutchison (
Date: Thu Dec 16 1999 - 06:35:04 PST

On Thu, 16 Dec 1999, Jens Moellenhoff wrote:

> browser by the program that builds these statistics. Is this an error of
> this program or can I change the config file of htdig, so that htdig
> shows up as a spider in these statistics?

I would say this is an error of that program. IMHO, there's little
difference between a spider and a browser--a spider is simply an automated
browser! I expcet it makes the distinction by keeping a list of known

You can also change the user-agent field of htdig if you choose. See

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Thu Dec 16 1999 - 06:48:59 PST