Re: [htdig] Following links, not indexing a doc


Subject: Re: [htdig] Following links, not indexing a doc
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Tue Nov 07 2000 - 09:51:32 PST


On Tue, 7 Nov 2000, Eric Bliss wrote:

> My problem is that although htdig's website says that it follows the
> robot rules, my index documents still show up when a search is done.
> Is there a different tag I should be using, or do you need to specify
> a setting in htdig for it to obey robot rules?

Are you sure you're using a recent version of ht://Dig? This functionality
was added in the 3.1 betas and there were a few bugs that cropped up
along the way as well. The latest stable version is 3.1.5:
<http://www.htdig.org/RELEASE.html>

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this. List archives: <http://www.htdig.org/mail/menu.html> FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Tue Nov 07 2000 - 09:58:19 PST