Re: [htdig] Going for the big dig

Subject: Re: [htdig] Going for the big dig
From: Geoff Hutchison (
Date: Mon Dec 18 2000 - 17:41:21 PST

At 10:14 AM +1100 12/19/00, Terry Collins wrote:
>And make sure you don't ignore robots.txt

Yes, though someone would need to alter the code to do this.

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Mon Dec 18 2000 - 19:25:08 PST