Re: [htdig] Indexing URLs


Subject: Re: [htdig] Indexing URLs
From: Vincent Queru (vqueru@zaphod.grenoble.hp.com)
Date: Wed Sep 27 2000 - 01:52:55 PDT


Torsten Neuer wrote:

> Vincent Queru wrote:
> >
> > But I still have one more question : I had included a META NAME="robots"
> > VALUE=noindex" tag in the page containing the links but they still got indexed, is
> > that normal ?
>
> Yes. You will achieve the intended behaviour with
> <META NAME="robots" CONTENT="noindex">

In fact I made a mistake when typing the previous mail and I use CONTENT="noindex" on
my site.

Anyway using the description_factor=0 DID solve the problem, so I won't bother you any
more with my questions unless you think there might be a problem somewhere else :

The URLs encountered all contain numbers and as I turned the number indexing on, this
is probably the reason why they got indexed in first place although they do not appear
as such between the <A HREF> and </A> tag.

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Wed Sep 27 2000 - 01:56:42 PDT