Re: htdig: Pages get indexed several times

Jesse op den Brouw (
Fri, 28 Nov 1997 12:57:18 +0100

Martin Berli wrote:
> (Looks like I sent this to the wrong address: seems not
> to work)
> I noted a problem with htdig: When indexing a site, it doesn't see the
> symlinks (coming via http), so search results return the same pages more than
> once. Does anyone have a solution to this? Maybe some postprocessing of htdig?

It doesn't see any of those symlinks because htdig gets it info on a
document basis, i.e. every URL is a document to htdig. If you want to
eliminate these links, the webmaster has to delete them.
> Thanks for any hints,
> Martin Berli

Correct me if I'm wrong....
J. op den Brouw Johanna Westerdijkplein 75
Haagse Hogeschool 2521 EN DEN HAAG
Sector Techniek Netherlands
Afdeling Elektrotechniek +31 70 4458936
-------------------- ---------------------

Linux - because reboots are for hardware changes
To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:25:13 PST