Re: htdig: meta robots tag problem


Mark Malecha (mark@pathology.utmem.edu)
Tue, 22 Sep 1998 09:37:02 -0500


Geoffrey.R.Hutchison@williams.edu said:
> ><meta name="robots" content="index,nofollow"> > >However, htsearch
> doesn't report the pages with this meta tag (I >assume they were not
> indexed, right?). Is this htdig bug, or >mistake on my part? What is
> a correct way of achieving the >desire effect?

> That *should* work. To see what's going on, try:

> <meta name="robots" content="nofollow"> and see if you see the same
> behavior.

I see exactly the same happening for the pages with
<meta name="robots" content="nofollow">, they are not being
indexed.

                    --------- Mark Malecha
                    --------- mark@pathology.utmem.edu

----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:49 PST