Re: [htdig] Going for the big dig

Subject: Re: [htdig] Going for the big dig
From: Terry Collins (
Date: Mon Dec 18 2000 - 15:14:08 PST

> > 1) Is ht://Dig able to efficiently index external sites?

> Depends mostly on the response of the site and the speed of your network
> connection! It's also usually a very good idea to be a good neighbor and
> ask the sysadmin of the box as far as typical low-usage periods, set
> server_wait_time high, etc.

And make sure you don't ignore robots.txt

   Terry Collins {:-)}}} Ph(02) 4627 2186 Fax(02) 4628 7861  
   email:  www:  
   WOA Computer Services <lan/wan, linux/unix, novell>

"People without trees are like fish without clean water"

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Mon Dec 18 2000 - 16:25:20 PST