Re: [htdig] Indexing URLs

Subject: Re: [htdig] Indexing URLs
From: Vincent Queru (
Date: Wed Sep 27 2000 - 01:52:55 PDT

Torsten Neuer wrote:

> Vincent Queru wrote:
> >
> > But I still have one more question : I had included a META NAME="robots"
> > VALUE=noindex" tag in the page containing the links but they still got indexed, is
> > that normal ?
> Yes. You will achieve the intended behaviour with
> <META NAME="robots" CONTENT="noindex">

In fact I made a mistake when typing the previous mail and I use CONTENT="noindex" on
my site.

Anyway using the description_factor=0 DID solve the problem, so I won't bother you any
more with my questions unless you think there might be a problem somewhere else :

The URLs encountered all contain numbers and as I turned the number indexing on, this
is probably the reason why they got indexed in first place although they do not appear
as such between the <A HREF> and </A> tag.

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.
List archives: <>
FAQ: <>

This archive was generated by hypermail 2b28 : Wed Sep 27 2000 - 01:56:42 PDT