Re: [htdig] Update search indexes?


Subject: Re: [htdig] Update search indexes?
From: Geoff Hutchison (ghutchis@wso.williams.edu)
Date: Mon Jul 24 2000 - 19:07:56 PDT


At 7:25 PM -0500 7/24/00, David Gibbs wrote:
>Would more memory effect the processing significantly? Maybe more
>disk? (got 6gb now). Is there a mode of operation where it only
>looks at changed pages for indexing?

Sure. If you have pre-existing databases then it will use those to
only look at changed pages. For example, the rundig.sh script
<http://www.htdig.org/files/contrib/scripts/> uses pre-existing .work
files to do this. (Granted, I'm a bit partial to this script. :-)

More memory or a faster disk might help. Also, if you have access to
the files through a local filesystem (preferably not NFS), see the
local_urls attributes:

e.g.
<http://www.htdig.org/attrs.html#local_urls>

--
-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Mon Jul 24 2000 - 09:22:23 PDT