Geoff Hutchison (email@example.com)
Fri, 29 Jan 1999 23:43:57 -0400
>seen. Does this version of htdig support ssi's? If not Any patches? Any
If you're indexing through HTTP, ht://Dig couldn't care whether it's SSI,
ASP, CGI, or whatever. All it will look at over HTTP is the end-result file
that the user would see.
>don't have any exclusions for htdig. Is it possible the larger web
>hosting's company server has a htdig exclusion? Similarly I didn't put
That shouldn't matter. When indexing, ht://Dig (or any standard indexer)
will grab /robots.txt before starting on the specific server. So it's your
virtual host that determines the /robots.txt file.
>any exclusions on indexing .shtml files in the htdig.conf file. I also
>tried deleting htdig and reinstalling it on the server and it that was
One of the best ways to see what ht://Dig is up to is to add debugging output.
htdig -vvv (flags of your choice)
This will generate a bunch of messages, including results of parsing
robots.txt, and HTTP headers.
Williams Students Online
To unsubscribe from the htdig mailing list, send a message to
firstname.lastname@example.org containing the single word "unsubscribe" in
the SUBJECT of the message.
This archive was generated by hypermail 2.0b3 on Sun Jan 31 1999 - 10:43:20 PST