htdig: Unable to contact server corrupts htdig run


Dane Jasper (dane@sonic.net)
Mon, 13 Oct 1997 16:33:58 -0700


We've got a very busy web server, and it is often restarted. While
trying to index the site, this causes problems. The site has about
60,000 pages, and an index run takes from four to six hours, and it's
gotten to the point where it's impossible to get a complete index.

I'm using 3.0.8b1 with Pasi's wonderful patches. Local file access is
not an option as we need htaccess style access controls in place. While
the server is restarting, I get thousands of failed accesses which look
like:

Unable to contact server: http://www.sonic.net/~josephb/frmain.htm Ref:
http://www.sonic.net/~josephb/
Unable to contact server: http://www.sonic.net/~iet/ncpt2.html Ref:
http://www.sonic.net/~iet/
Unable to contact server: http://www.sonic.net/~iet/fasting.html Ref:
http://www.sonic.net/~iet/

The server is typicly offline for about one and a half minutes every
6-12 hours during a restart. As it hosts about 300 multihomes,
restarting takes a long time!

I'd like it if ht://Dig would wait a configurable amount of seconds and
retry a configurable number of times before giving up. Any ideas? A
response via email would be appreciated, as I'm not on the list at this
time.

--
Dane Jasper
Sonic
----------------------------------------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-request@sdsu.edu containing the single word "unsubscribe" in
the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:25:10 PST