htdig: Feature request

Michael Graff (
14 Sep 1998 14:31:28 -0700

There is a way to limit what pages are indexed in htdig, using meta
tags. I propose an additional flag that says "collect URLs from this
page, but do NOT index this page."

I'd use this for mailing list archives to avoid indexing the list
index page, but to gather all the contained URLs.

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:43 PST