Re: htdig: exclude_urls


Geoff Hutchison (Geoffrey.R.Hutchison@williams.edu)
Mon, 05 Oct 1998 14:49:58 -0400


>Can "exclude_urls" be used to exclude one host from my domain
>when I am htdigging my whole domain (limit_urls_to: .mydomain.com)?
>Will the following work?
>exclude_urls: http://badhost.mydomain.com

It should.

>exclude the url (badhost) based on the ip address? Literal

No. Exclude_urls is done based on the string matching. So if the strings
are not identical, the URL will not be excluded. You can either include all
the aliases in exclude_urls or add a robots.txt file to
badhost.mydomain.com that restricts ht://Dig from accessing files on the
server. This will be read no matter what alias is used.

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:28 PST