Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1

Subject: Re: [htdig3-dev] [ANNOUNCE] ht://Dig 3.2.0b1
From: Valdas Andrulis (
Date: Sat Feb 05 2000 - 12:29:42 PST

I am searching my site. I have put in my robots.txt:

User-agent: *
Disallow: /cat/
User-agent: htdig
Disallow: /cat/

but htdig doesn't pay attention on that, it still indexes pages like

maybe i am doing something wrong?


To unsubscribe from the htdig3-dev mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Sat Feb 05 2000 - 12:31:46 PST