Martin Pohl (email@example.com)
Mon, 22 Feb 1999 15:34:07 +0100
Yann Doussot wrote:
> I've just installed htdig 3.1.1 on a Linux server. Works fine
> except the META TAGS. I've added <META NAME="robots" VALUE="noindex"> to
> several pages, rebuilt database but these are still returned by a search.
> Ideas ? May be I missunderstand the meaning of the tag (I want links in
> the page to be followed but I don't want to index it).
As far as I know, your Meta-tag is only supported by very few robots.
Obviously not by ht://Dig. Another method would be to create a file
'robot.txt' in the root directory of the WWW server, which contains
User-agent:<robot-name> | "*"
[Allow: /path/[file.html] ]
All lines can appear more than once. If you use the asterisk in the
first line, all robots should regard the exclusions.
Let me know, if it had worked.
-- _____________________________________________________ Dr. Martin Pohl Institute for Orthopaedic Research and Biomechanics Helmholtzstr. 14 89081 ULM, Germany, EU phone: +49-(0)731-50-23494 / FAX: +49-(0)731-50-23498 WWW: http://pollux.medizin.uni-ulm.de e-mail: mailto://firstname.lastname@example.org _____________________________________________________ ------------------------------------ To unsubscribe from the htdig mailing list, send a message to email@example.com containing the single word "unsubscribe" in the SUBJECT of the message.
This archive was generated by hypermail 2.0b3 on Mon Feb 22 1999 - 07:08:23 PST