Re: [htdig] Quick question

Subject: Re: [htdig] Quick question
From: Doug Barton (
Date: Sun Mar 05 2000 - 11:55:18 PST

"Glenn J. Rowe" wrote:
> Pardon me. I just started using htdig and just now joined this mailing
> list. I have a question which I am sure someone will be able to answer.
> I have specified a rather small list of sites that should be indexed.
> htdig does only index those sites; however, when indexing it follows
> links to sites that aren't in the list. This poses a problem because a
> few sites have a large amount of external links on them and htdig
> follows everyone of those links. It doesn't index them but it follows
> them thus making the indexing process take FOREVER. Is there a way to
> stop that?

        What is the limit_urls set to in your htdig.conf?


"Welcome to the desert of the real." 

- Laurence Fishburne as Morpheus, "The Matrix"

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Sun Mar 05 2000 - 11:59:56 PST