Re: htdig: Fixed DB bug (problems)

Reiner Keller (
Mon, 2 Nov 1998 13:22:05 +0100 (MET)


> The patch fixing the database should have no effect whatsoever on which
> URLs are found--none of that code changed!
> I suggest doing a dig with "htdig -vvv" and looking at what URLs it says it
> rejects.

htdig looks only for the first document. Because it isn't changed, htdig
seems to think that the other documents haven't changed, too...


New server:, 80
Retrieval command for GET /robots.txt HTTP/1.0
User-Agent: htdig/3.1.0b1 (

Header line: HTTP/1.1 200 OK Header line: Date: Mon, 02 Nov 1998 02:00:02 GMT Header line: Server: Apache/1.3.1 Ben-SSL/1.22 (Unix) mod_perl/1.15 PHP/3.0.3 Header line: Last-Modified: Sat, 16 May 1998 01:01:49 GMT Translated Sat, 16 May 1998 01:01:49 GMT to Sat, 16 May 1998 01:01:49 (98) And converted to Sat, 16 May 1998 01:01:49 Header line: ETag: "28064-3a-355ce57d" Header line: Accept-Ranges: bytes Header line: Content-Length: 58 Header line: Connection: close Header line: Content-Type: text/plain Header line: returnStatus = 0 Read 58 from document Read a total of 58 bytes Parsing robots.txt file using myname = htdig Robots.txt line: #fuer alle Robots gilt: Robots.txt line: User-agent: * Found 'user-agent' line: * Robots.txt line: Disallow: /intern/ Found 'disallow' line: /intern/ Pattern: /intern/ pick:, # servers = 1 0:0:0: Trying local file /home/www/index.html not changed pick:, # servers = 1 1:0:0: Trying local file /home/www/index.html not changed pick:, # servers = 1 htdig: Run complete htdig: 1 server seen: htdig: 2 documents htmerge: Total word count: 54429 htmerge: Total documents: 1 htmerge: Total doc db size (in K): 5

------------------------------------------------------------------------ Reiner Keller e-mail: WWW : ------------------------------------------------------------------------ ---------------------------------------------------------------------- To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:43 PST