Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1


Subject: Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Sat Feb 05 2000 - 16:50:11 PST


At 10:29 PM +0200 2/5/00, Valdas Andrulis wrote:
>User-agent: *
>Disallow: /cat/
>User-agent: htdig
>Disallow: /cat/
>
>but htdig doesn't pay attention on that, it still indexes pages like
>/cat/add_new_entry.php3?cat_id=121

This is a bit strange. The robots.txt you mentioned is redundant--it
says "no one should index /cat/" and then "htdig should not index
/cat/"

Nevertheless, it shouldn't do this. What version is this? 3.2.0b1?
Does it seem to work under 3.1.4?

Thanks,

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
htdig3-dev-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Sat Feb 05 2000 - 16:53:12 PST