Re: [htdig] Rate limiting spidering?


Subject: Re: [htdig] Rate limiting spidering?
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Thu Aug 10 2000 - 15:04:38 PDT


On Thu, 10 Aug 2000, Albert Lunde wrote:

> Is there a way to control something like the number of hits per
> minute htdig makes on a give server?

In the production code, no. In the 3.2 code, the config parser allows
per-server and per-URL attributes, though not all attributes are "turned
on." (Unfortunately, the 3.2 documentation indicates the current wishlist
and not whether they actually work this way yet.)

But to answer your question, yes server_wait_time is among those that
works in this fashion in the current 3.2 code. Unfortunately there's no
way to do it in 3.1.x w/o multiple config files.

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Thu Aug 10 2000 - 05:04:41 PDT