Re: [htdig] HTDIG doesn't follow /cgi-bin/name.cgi?option=blafasel

Geoff Hutchison (
Wed, 17 Mar 1999 11:02:01 -0500 (EST)

On Wed, 17 Mar 1999, gstammw wrote:

> What can I do ??

Either change the config to only exclude CGIs from your host, or run two
passes, one on your host, one on external hosts. One config will be your
current and one will have the CGIs included (or not excluded...)

> Another question: is there the possibility to disallow some hosts like
> or something like that :-) I don't want all these
> porno-shit-sites in my index.

See exclude_urls... :-)

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Wed Mar 17 1999 - 10:05:13 PST