Subject: Re: [htdig] Going for the big dig
From: Terry Collins (terryc@woa.com.au)
Date: Mon Dec 18 2000 - 15:14:08 PST
> > 1) Is ht://Dig able to efficiently index external sites?
> Depends mostly on the response of the site and the speed of your network
> connection! It's also usually a very good idea to be a good neighbor and
> ask the sysadmin of the box as far as typical low-usage periods, set
> server_wait_time high, etc.
And make sure you don't ignore robots.txt
--
Terry Collins {:-)}}} Ph(02) 4627 2186 Fax(02) 4628 7861
email: terryc@woa.com.au www: http://www.woa.com.au
WOA Computer Services <lan/wan, linux/unix, novell>
"People without trees are like fish without clean water"
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>
This archive was generated by hypermail 2b28 : Mon Dec 18 2000 - 16:25:20 PST