Re: htdig: There has to be an easier way

Geoff Hutchison (
Tue, 03 Nov 1998 08:54:43 -0500

At 10:16 AM -0500 10/24/98, Wayne Spivak wrote:
>I can't for the life of me get Htdig to dig though the entire tree.
>These are the pertinent parts of my config file:

I would run "htdig -vvv" and redirect this to a file. Then look through
this file to see why it is rejecting certain URLs. My first impression is
that since you're using the local_urls, that it's rejecting files that
don't end in .html since it can't parse .cgi files.

-Geoff Hutchison
Williams Students Online

>database_dir: /www/htdig/db_com
>exclude_urls: /cgi-bin/
>max_head_length: 60000
>search_algorithm: exact:1 synonyms:0.5 endings:0.1 soundex:0.5
>allow_numbers: true
>allow_virtual_hosts: true
>create_url_list: true
>url_list: /www/htdig/url_list_com
>timeout: 42

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:44 PST