Re: [htdig] htdig won't ignore the files I want ignored


Subject: Re: [htdig] htdig won't ignore the files I want ignored
From: Michael Johnson (vwbug@southwind.net)
Date: Thu Dec 16 1999 - 14:07:09 PST


Try listing the full URL to be excluded, i.e.
http://raven.cc.ukans.edu/~kups/maillist/polsannounce

I don't know that this will fix it, I bet it will.

You can also use a robots.txt file. Take a look at
http://info.webcrawler.com/mak/projects/robots/norobots.html
for info on how to create one.

---------------------------------------------------------------------
SouthWind Internet Access, Inc. 120 S MarketSt. Suite 300
Michael Johnson (M.J.) Wichita, KS 67202
---------------------------------------------------------------------
            Ignorance is not knowing that you don't know.

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Thu Dec 16 1999 - 14:20:54 PST