Re: [htdig] A few points.


Subject: Re: [htdig] A few points.
From: gil cohen (cohengil4@hotmail.com)
Date: Mon May 15 2000 - 16:33:45 PDT


Could you please name a few of the perl scrtips? I went hunting around a
little bit and couldn't find any.

>From: Geoff Hutchison <ghutchis@wso.williams.edu>
>To: "gil cohen" <cohengil4@hotmail.com>
>CC: htdig@htdig.org
>Subject: Re: [htdig] A few points.
>Date: Mon, 15 May 2000 17:46:30 -0500
>
>At 6:22 PM -0400 5/15/00, gil cohen wrote:
>>Is there any way to benchmark htdig? Performance seems to degrade as
>>it crawls through more and more URLs. I've tried to use db.urls as a
>>way to benchmark,
>
>Performance is not linear. So it will slow down as the number of URLs
>increase. If you want to test this, I'd probably use the -v flag to
>spit up a nice progress report.
>
>>If it is true that htdig slows down after a while, do any of you
>>know anyway to make it faster? (I honestly don't care about the
>>searching use for htdig. All I'm using it for is getting a list of
>>unique URLs from a site, and passing it through my parser)
>
>This seems like a rather inefficient way of doing it. There are any
>number of Perl scripts that will happily do this for you. Of course
>they don't provide any searching functionality, which in your case
>would be a "freebie."
>
>Cheers,
>
>--
>-Geoff Hutchison
>Williams Students Online
>http://wso.williams.edu/
>
>------------------------------------
>To unsubscribe from the htdig mailing list, send a message to
>htdig-unsubscribe@htdig.org
>You will receive a message to confirm this.
>

________________________________________________________________________
Get Your Private, Free E-mail from MSN Hotmail at http://www.hotmail.com

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Mon May 15 2000 - 14:22:16 PDT