[htdig] Indexed at depth 255!

Subject: [htdig] Indexed at depth 255!
From: Malcolm Austen (malcolm@sable.ox.ac.uk)
Date: Fri Nov 05 1999 - 01:48:28 PST

I've held off querying this until I got around to installing the 3.1.3_0
rpm (i386/RedHat 5.2). The oddity didn't go away so ...

I index 138 servers across the University, some pages on just two of those
servers get logged by htdig as having been indexed at depth 255! This is
with -h 15 so I suspect the depth is actually "-1". I've looked at the
output from htdig (-vvv) and can't see anything that indicates why this is
happening so it may be a variable being primed to "-1" and then never
being updated?

This only came to light when I modified the "-s" scanning perl script that
Daniel MacKay posted. My modified script constructs a table of how many
pages were indexed at depth "n" on each server. That in itself was
interesting, most servers have everything within about 6-7-8 clicks of the
top level but we actually have a handful of servers with pages that can
only be reached with 15 (or more) mouse clicks!

The report from my hack of Daniel's perl script can be seen at


and I've placed a copy of the script at


if anyone is interested.


To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.

This archive was generated by hypermail 2b25 : Fri Nov 05 1999 - 01:58:31 PST