Re: [htdig] problem indexing a site

Subject: Re: [htdig] problem indexing a site
From: Geoff Hutchison (
Date: Wed Jan 17 2001 - 07:34:23 PST

On Wed, 17 Jan 2001, Elsa Chan wrote:

> not indicate any errors. It looks like as follows:
> 1:0:
> New server:, 80
> It just sits there for a long while.

The first thing you should check is if you can contact this site with
another browser, e.g. lynx, Netscape, etc. The first thing htdig must
do is to retrieve the robots.txt file from the server. So if you cannot
connect to the server using other means, htdig will not be able to either
and you will have to look at networking issues.

That said, it should not just "hang" since there is a timeout set in the
connection code and the 3.1.5 version should be good about killing
connections if they timeout. How long is "a long while?"

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Wed Jan 17 2001 - 07:49:07 PST