Re: [htdig] exclude question

Gilles Detillieux (
Thu, 6 May 1999 14:49:28 -0500 (CDT)

According to Eric Luhrs:
> My start_url is a simple list of pointers to each of the pages I want
> htdig to index. The problem is that I don't want this page in htdig's
> database. I have tried to restrict it with exclude_url and I have also
> put it on a server which is not in limit_urls_to, but it keeps showing up
> in my searches. Any ideas how to to exclude start_url from indexes?

I think in general, you can put

        <meta name=robots content=noindex>

at the start of a document, and htdig will still follow links in the
document, but won't index the document itself.

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930
To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Thu May 06 1999 - 12:59:58 PDT