Re: [htdig] Some pages are parsed very slowly.


Subject: Re: [htdig] Some pages are parsed very slowly.
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Thu May 11 2000 - 12:14:20 PDT


On Thu, 11 May 2000, NEPOTE Charles (Neuilly Gestion) wrote:

>
> I discovered that some pages needs very high CPU activity to be parsed by
> htdig.
> These pages are all about 40 Ko long and contains about 3 or 4 dozen of
> links such as folowing :
> http://intra.xxx.zzz/docu/weblib/perl/ch02_05.htm#PERL2-CH-2-SECT-5.10.
>
> Why ? Maybe the sharp (#) ?

It wouldn't be the sharp because this is ignored on the URL parser.
However, pages with 3 or 4 dozen links could push things up because each
link will need to be checked against the database.

I guess my first question would be if you have pages of 3-4 dozen links
without any sort of # mark. This would provide some element of comparison.

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Thu May 11 2000 - 10:01:51 PDT