Gilles Detillieux (grdetil@scrc.umanitoba.ca)
Thu, 6 May 1999 14:49:28 -0500 (CDT)
According to Eric Luhrs:
> My start_url is a simple list of pointers to each of the pages I want
> htdig to index. The problem is that I don't want this page in htdig's
> database. I have tried to restrict it with exclude_url and I have also
> put it on a server which is not in limit_urls_to, but it keeps showing up
> in my searches. Any ideas how to to exclude start_url from indexes?
I think in general, you can put
<meta name=robots content=noindex>
at the start of a document, and htdig will still follow links in the
document, but won't index the document itself.
-- Gilles R. Detillieux E-mail: <grdetil@scrc.umanitoba.ca> Spinal Cord Research Centre WWW: http://www.scrc.umanitoba.ca/~grdetil Dept. Physiology, U. of Manitoba Phone: (204)789-3766 Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930 ------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig@htdig.org containing the single word "unsubscribe" in the SUBJECT of the message.
This archive was generated by hypermail 2.0b3 on Thu May 06 1999 - 12:59:58 PDT