[htdig] Indexed at depth 255!


Subject: [htdig] Indexed at depth 255!
From: Malcolm Austen (malcolm@sable.ox.ac.uk)
Date: Fri Nov 05 1999 - 01:48:28 PST


I've held off querying this until I got around to installing the 3.1.3_0
rpm (i386/RedHat 5.2). The oddity didn't go away so ...

I index 138 servers across the University, some pages on just two of those
servers get logged by htdig as having been indexed at depth 255! This is
with -h 15 so I suspect the depth is actually "-1". I've looked at the
output from htdig (-vvv) and can't see anything that indicates why this is
happening so it may be a variable being primed to "-1" and then never
being updated?

This only came to light when I modified the "-s" scanning perl script that
Daniel MacKay posted. My modified script constructs a table of how many
pages were indexed at depth "n" on each server. That in itself was
interesting, most servers have everything within about 6-7-8 clicks of the
top level but we actually have a handful of servers with pages that can
only be reached with 15 (or more) mouse clicks!

The report from my hack of Daniel's perl script can be seen at

        http://daneel.oucs.ox.ac.uk/wwwsearch/report/

and I've placed a copy of the script at

        http://daneel.oucs.ox.ac.uk/wwwsearch/report/dig_report.pl

if anyone is interested.

regards,
        Malcolm.
                 Malcolm.Austen@oucs.ox.ac.uk

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.



This archive was generated by hypermail 2b25 : Fri Nov 05 1999 - 01:58:31 PST