Subject: Re: [htdig] Indexing URLs
From: Vincent Queru (firstname.lastname@example.org)
Date: Wed Sep 27 2000 - 01:52:55 PDT
Torsten Neuer wrote:
> Vincent Queru wrote:
> > But I still have one more question : I had included a META NAME="robots"
> > VALUE=noindex" tag in the page containing the links but they still got indexed, is
> > that normal ?
> Yes. You will achieve the intended behaviour with
> <META NAME="robots" CONTENT="noindex">
In fact I made a mistake when typing the previous mail and I use CONTENT="noindex" on
Anyway using the description_factor=0 DID solve the problem, so I won't bother you any
more with my questions unless you think there might be a problem somewhere else :
The URLs encountered all contain numbers and as I turned the number indexing on, this
is probably the reason why they got indexed in first place although they do not appear
as such between the <A HREF> and </A> tag.
To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
This archive was generated by hypermail 2b28 : Wed Sep 27 2000 - 01:56:42 PDT