Subject: [htdig3-dev] ht://Dig logging and reporting
From: Tom Metro (tmetro@vl.com)
Date: Wed Nov 17 1999 - 13:20:09 PST
htdig doesn't appear to offer any options for logging indexed
documents. Piping the output from -v to a file would be one way, but
not very nicely formatted. Has any thought been given to having htdig
generate a Common Log Format (CLF) log file?
It would be nice to have some reporting tools. I haven't tried htdig
-s, but it would be nice to be able to get database stats without
having to perform a dig, and what I'd really like would be a tool that
could produce a list of indexed documents after-the-fact (so a
database generated by an overnight cron job could be examined).
I'm sure this could be hacked together by converting the database to
ASCII and then filtering it with Perl, but that could be prohibitive
if you were low on disk space and also sounds like an excessive amount
of work just to get the list of indexed documents.
I would think that being able to easily track which documents have
been "dug" would be critical for determining whether htdig's parser is
having difficulty following links (either due to parser bugs or more
likely HTML problems) and is leaving orphaned documents behind. A
check that all administrators would want to perform on new
installations.
-Tom
-- Tom Metro Venture Logic tmetro@vl.com Newton, MA, USA------------------------------------ To unsubscribe from the htdig3-dev mailing list, send a message to htdig3-dev-unsubscribe@htdig.org You'll receive a message confirming the unsubscription.
This archive was generated by hypermail 2b25 : Wed Nov 17 1999 - 13:32:59 PST