Re: [htdig] Suse 6.2 + htdig 3.1.5: looping again


Subject: Re: [htdig] Suse 6.2 + htdig 3.1.5: looping again
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Mon May 01 2000 - 16:10:43 PDT


At 5:11 PM -0500 5/1/00, Gilles Detillieux wrote:
> > I make some simple assumptions here, but if only a hash table of visited
> > URLs is stored in memory, to avoid having to open the db file of urls
> > every time, then this would not slow things down too much ?

I'll echo what Gilles said again. What you describe is *exactly* what
htdig uses. There are a variety of issues if you're browsing through
the Retriever code, but suffice to say there is a hash of the visited
URLs.

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Mon May 01 2000 - 15:37:10 PDT