Re: htdig: Missing Files in Search Results

Geoff Hutchison (
Sun, 08 Nov 1998 09:58:21 -0500 (EST)

> I've taken a file that was indexed previously and simply copied it to
> another file name. The file has exactly the same contents, exactly the
> same permissions. Htdig indexes the old file, not the new one. Htdig is
> indexing files of a newer date as I've got some recent additions that
> are indexed.

Sure. It won't index the new one unless there's a link to the new file as
well. Unlike some spiders (notably SWISH), ht://Dig will only follow links
from some source document, the start_url. So if there isn't a link, it
can't find it.

> Best I can see, it's arbitrarily skipping files. But of course, it can't
> be arbitrary.

Well if the above explanation sounds reasonable, check to see if there are
links to everything. If there *are* and it's skipping them... that's a

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:45 PST