[htdig] SSL site index


Subject: [htdig] SSL site index
From: Walls Rob W MSgt 75 CS/SCX (Rob.Walls@HILL.af.mil)
Date: Tue Aug 29 2000 - 16:56:34 PDT


I have found htdig to be the fastest and most configurable indexer around.
Now, just like everyone else it seems, my site must go secure. This
question has been asked many times, but I'll ask it again...

Can htdig index a secure site?

One answer to this question from Browny said that htdig would now index
https sites in version 3.1.4. I have 3.1.4. and htdig won't talk SSL.
Would recompiling htdig with the openssl libs fix this?
I am running Apache and the entire site is using SSL on port 443 (not even
listening on port 80)

Doing an "htdig -i -vvv" returns:
------------------------------------------------------
[root]:/web/htdig/bin >./htdig -i -vvv
        1:0:"https://commweb.hill.af.mil:443/"
New server: commweb.hill.af.mil, 443
Retrieval command for http://commweb.hill.af.mil:443/robots.txt: GET
/robots.t
xt HTTP/1.0
User-Agent: htdig/3.1.4 (root@commweb.hill.af.mil)
Host: commweb.hill.af.mil

Header line: HTTP/1.1 403 Forbidden
Header line: Date: Tue, 29 Aug 2000 23:17:44 GMT
Header line: Server: Apache/1.3.12 (Unix) mod_perl/1.24 mod_ssl/2.6.4
OpenSSL/
0.9.5a
Header line: Connection: close
Header line: Content-Type: text/html; charset=iso-8859-1
Header line:
returnStatus = 1
 pushed
pick: commweb.hill.af.mil, # servers = 1
0:0:0:"https://commweb.hill.af.mil:443/": Retrieval command for
"https://commw
eb.hill.af.mil:443/": GET /" HTTP/1.0
User-Agent: htdig/3.1.4 (root@commweb.hill.af.mil)
Host: commweb.hill.af.mil

Header line: HTTP/1.1 403 Forbidden
Header line: Date: Tue, 29 Aug 2000 23:17:44 GMT
Header line: Server: Apache/1.3.12 (Unix) mod_perl/1.24 mod_ssl/2.6.4
OpenSSL/
0.9.5a
Header line: Connection: close
Header line: Content-Type: text/html; charset=iso-8859-1
Header line:
returnStatus = 1
 not found
pick: commweb.hill.af.mil, # servers = 1
--------------------------------------------------------

Apache's error log contains the related errors:

--------------------------------------------------------
[Tue Aug 29 17:17:44 2000] [error] mod_ssl: SSL handshake failed: HTTP
spoken
on HTTPS port; trying to send HTML error page (OpenSSL library error
follows)
[Tue Aug 29 17:17:44 2000] [error] OpenSSL: error:1407609C:SSL
routines:SSL23_
GET_CLIENT_HELLO:http request [Hint: speaking HTTP to HTTPS port!?]
[Tue Aug 29 17:17:44 2000] [error] access to
/web/htdocs/mod_ssl:error:HTTP-re
quest failed for commweb.hill.af.mil, reason: SSL connection required
[Tue Aug 29 17:17:44 2000] [error] (2)No such file or directory: access to
/we
b/htdocs/error/403.htm failed for commweb.hill.af.mil, reason: SSL
connection
required
[Tue Aug 29 17:17:44 2000] [error] mod_ssl: SSL handshake failed: HTTP
spoken
on HTTPS port; trying to send HTML error page (OpenSSL library error
follows)
[Tue Aug 29 17:17:44 2000] [error] OpenSSL: error:1407609C:SSL
routines:SSL23_
GET_CLIENT_HELLO:http request [Hint: speaking HTTP to HTTPS port!?]
[Tue Aug 29 17:17:44 2000] [error] access to
/web/htdocs/mod_ssl:error:HTTP-re
quest failed for commweb.hill.af.mil, reason: SSL connection required
[Tue Aug 29 17:17:44 2000] [error] (2)No such file or directory: access to
/we
b/htdocs/error/403.htm failed for commweb.hill.af.mil, reason: SSL
connection
required
------------------------------------------------------

So, does htdig do SSL and if so how?
I would really hate to have to quit using the world's best indexer!

Also, it seems a seperate port directive would be nice, unless the port is
extracted from the start_url and used for every request (the links in the
pages don't specify port #)

Thanks,

MSgt Rob W. Walls
75CS/SCX
DSN 777-2230
CML (801) 777-2230
rob.walls@hill.af.mil

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig-unsubscribe@htdig.org
You will receive a message to confirm this.
List archives: <http://www.htdig.org/mail/menu.html>
FAQ: <http://www.htdig.org/FAQ.html>



This archive was generated by hypermail 2b28 : Tue Aug 29 2000 - 16:58:06 PDT