Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1


Subject: Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1
From: Valdas Andrulis (valdand@soften.ktu.lt)
Date: Sat Feb 05 2000 - 12:29:42 PST


I am searching my site. I have put in my robots.txt:

User-agent: *
Disallow: /cat/
User-agent: htdig
Disallow: /cat/

but htdig doesn't pay attention on that, it still indexes pages like
/cat/add_new_entry.php3?cat_id=121

maybe i am doing something wrong?

VAldas

------------------------------------
To unsubscribe from the htdig3-dev mailing list, send a message to
htdig3-dev-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Sat Feb 05 2000 - 12:31:46 PST