htdig: Re: patch for new urls

Edmond Abrahamian (
Wed, 22 Apr 1998 21:01:19 +0200 (EET)

On Wed, 22 Apr 1998, Steve Scott [UUcom] wrote:

> I made a html page and digged the database. After I edited
> the first page and added a link to a second page. When I
> redigged with the -n and -a options, the second page was never
> included in the database. I could only get it added to the
> database if I digged with the -i option. This seems to cancel out
> the purpose of your patch. Is your patch running with the -a option and
> working as you described?

It should work with or without the -a option. I have also double-checked
this here. The most likely reason why your new link was not followed is the
value of ${limit_new_urls_to} which is a trap I have fallen into earlier

My test consisted of a simple site all the urls to which are contained in
one document. ONE: I build the db with -i. TWO: I change the root document and
add yet one more url in it. THREE: I re-dig the root document with -n, by
specifying the values of ${new_url} and ${limit_new_urls_to} in the config
file as well. At this point, htdig will recognize that the doc has changed
and will re-parse the urls, and attempt to get the new one only if it matches
in ${limit_new_urls_to}


  -- edmond

p.s. thanks for your comments. this is what we look for in beta work.
To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:03 PST