Frank Guangxin Liu (firstname.lastname@example.org)
Thu, 30 Sep 1999 13:53:51 -0500 (EST)
On Thu, 30 Sep 1999, Geoff Hutchison wrote:
> On Thu, 30 Sep 1999, Frank Guangxin Liu wrote:
> > Today I found htdig failed to index a site which requires no
> > authentication. To narrow the problem, I set
> It *doesn't* require authentication?
> > Header line: Server: Microsoft-IIS/4.0
> > it can index this site without a problem. Now it seems to
> > me htdig may need to do another try if password failed
> > by the server.
> But you see, this is a server bug. My reading of the HTTP standard is that
I agree it must be a server bug, or maybe it is the wrong config.
> it *can* send the authentication even if the URL doesn't require it. Most
> servers don't seem to have a problem with this--they ignore the
> Authentication header.
That's good, they just ignore.
> IIS apparently disagrees.
> The problem with trying again w/o a password is that this would only work
> for IIS servers. For everyone else, it would just be a redundant HTTP
> It's a tricky situation. Technically, browsers send a request, then
> realize they need the authentication, then send another response. This
> would solve the problem, but for any site that needs authentication, it
> would require twice as many HTTP connections. You might not notice for a
> browser, but when indexing hundreds or thousands of URLs, it'll be a
> performance hit.
I don't see why we need twice for all connections.
We only send twice if we got authentication failure. Most of
the time, we won't get a failure (as you stated above, the server
just ignores the header).
In the future, when the username/password/site config file (as I suggested
earlier) is implmented, htdig shouldn't
send auth header if a site isn't matched in the username/password/site
> -Geoff Hutchison
> Williams Students Online
> To unsubscribe from the htdig mailing list, send a message to
> email@example.com containing the single word unsubscribe in
> the SUBJECT of the message.
To unsubscribe from the htdig mailing list, send a message to
firstname.lastname@example.org containing the single word unsubscribe in
the SUBJECT of the message.
This archive was generated by hypermail 2.0b3 on Thu Sep 30 1999 - 12:01:13 PDT