Re: [htdig] Duplicate URLs?

Geoff Hutchison (
Mon, 28 Jun 1999 08:33:43 -0400

Jim Cole wrote:

> Is this normal behavior? Might it have something to do with why my
> databases are becoming so large?

This is not normal behavior. I would guess you have a symlink in those
directories. This symlink has essentially created an infinite loop. For
example, fyi -> fyi is one such loop right there.

This is almost definitely the root cause of large databases--you're
indexing your files *many*, *many* times over.

-Geoff Hutchison
Williams Students Online
To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Mon Jun 28 1999 - 04:53:41 PDT