htdig: meta robots tag problem


mark@pathology.utmem.edu
Fri, 18 Sep 1998 20:51:02 -0500


If I understand meta tag robots directive correctly the
following should index the page but disallow htdig from
following the links:

<meta name="robots" content="index,nofollow">

However, htsearch doesn't report the pages with this meta tag (I
assume they were not indexed, right?). Is this htdig bug, or
mistake on my part? What is a correct way of achieving the
desire effect?

TIA

                        -------- Mark Malecha
                        -------- mark@pathology.utmem.edu

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:49 PST