htdig: meta robots tag problem
Fri, 18 Sep 1998 20:51:02 -0500

If I understand meta tag robots directive correctly the
following should index the page but disallow htdig from
following the links:

<meta name="robots" content="index,nofollow">

However, htsearch doesn't report the pages with this meta tag (I
assume they were not indexed, right?). Is this htdig bug, or
mistake on my part? What is a correct way of achieving the
desire effect?


                        -------- Mark Malecha

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:49 PST