Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1


Subject: Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1
From: Valdas Andrulis (valdand@soften.ktu.lt)
Date: Sun Feb 06 2000 - 09:40:34 PST


On Sat, 5 Feb 2000, Geoff Hutchison wrote:

GH> At 10:29 PM +0200 2/5/00, Valdas Andrulis wrote:
GH> >User-agent: *
GH> >Disallow: /cat/
GH> >User-agent: htdig
GH> >Disallow: /cat/
GH> >
GH> >but htdig doesn't pay attention on that, it still indexes pages like
GH> >/cat/add_new_entry.php3?cat_id=121
GH>
GH> This is a bit strange. The robots.txt you mentioned is redundant--it
GH> says "no one should index /cat/" and then "htdig should not index
GH> /cat/"
GH>
GH> Nevertheless, it shouldn't do this. What version is this? 3.2.0b1?
GH> Does it seem to work under 3.1.4?

I doesnt work with 3.2.0b1, and do work with 3.1.4, with that redundant
robots.txt configurations, or just
User-agent: *
Disallow: /cat/

or

User-agent: htdig
Disallow: /cat/

seprately.

As i said i'll send logs later.

VAldas

GH>
GH> Thanks,
GH>
GH> -Geoff Hutchison
GH> Williams Students Online
GH> http://wso.williams.edu/
GH>
GH> ------------------------------------
GH> To unsubscribe from the htdig3-dev mailing list, send a message to
GH> htdig3-dev-unsubscribe@htdig.org
GH> You will receive a message to confirm this.
GH>
GH>
GH>

------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
htdig3-dev-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Sun Feb 06 2000 - 09:42:38 PST