Re: [htdig] Manually Add or Delete Documents from the Database ?

Geoff Hutchison (
Thu, 1 Jul 1999 21:27:47 -0400 (EDT)

On Sat, 19 Jun 1999, Michael Reutlinger wrote:

> The deleted files always generate a error message
> reported to the administrator as a broken Link
> by htDigs Update prozess. But thats not true, cause the
> Files are not been referenced by any of the index files.

On an update dig, ht://Dig starts off by checking all of the previous
URLs. But if you're getting "broken link" reports for these deleted files,
I would say that's a bug.

> Beside that htDig also add the static navigation files like
> index pages etc. to the database, which isn't what we would like
> to have in the database.

Try adding <meta name="robots" contents="noindex,follow"> tags to these
pages. These will tell ht://Dig and other robots that support the tags to
follow links but not index the pages.

> Is there a commandline flag to tell htdig to add one Document
> to his database or to delete one document from its database ??

Not at the moment. It seems to be a hot topic. :-)

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Thu Jul 01 1999 - 17:43:34 PDT