Re: htdig: Fixed DB bug (problems)


Reiner Keller (dlrg@cs.tu-berlin.de)
Mon, 2 Nov 1998 13:22:05 +0100 (MET)


Hello,

> The patch fixing the database should have no effect whatsoever on which
> URLs are found--none of that code changed!
>
> I suggest doing a dig with "htdig -vvv" and looking at what URLs it says it
> rejects.

htdig looks only for the first document. Because it isn't changed, htdig
seems to think that the other documents haven't changed, too...

Reiner

-- 
------------------------------------------------------------------------
New server: www.dlrg.de, 80
Retrieval command for http://www.dlrg.de/robots.txt: GET /robots.txt HTTP/1.0
User-Agent: htdig/3.1.0b1 (Webmaster@DLRG.de)
Host: www.dlrg.de

Header line: HTTP/1.1 200 OK Header line: Date: Mon, 02 Nov 1998 02:00:02 GMT Header line: Server: Apache/1.3.1 Ben-SSL/1.22 (Unix) mod_perl/1.15 PHP/3.0.3 Header line: Last-Modified: Sat, 16 May 1998 01:01:49 GMT Translated Sat, 16 May 1998 01:01:49 GMT to Sat, 16 May 1998 01:01:49 (98) And converted to Sat, 16 May 1998 01:01:49 Header line: ETag: "28064-3a-355ce57d" Header line: Accept-Ranges: bytes Header line: Content-Length: 58 Header line: Connection: close Header line: Content-Type: text/plain Header line: returnStatus = 0 Read 58 from document Read a total of 58 bytes Parsing robots.txt file using myname = htdig Robots.txt line: #fuer alle Robots gilt: Robots.txt line: User-agent: * Found 'user-agent' line: * Robots.txt line: Disallow: /intern/ Found 'disallow' line: /intern/ Pattern: /intern/ pick: www.dlrg.de:80, # servers = 1 0:0:0:http://www.dlrg.de/: Trying local file /home/www/index.html not changed pick: www.dlrg.de:80, # servers = 1 1:0:0:http://www.dlrg.de/: Trying local file /home/www/index.html not changed pick: www.dlrg.de:80, # servers = 1 htdig: Run complete htdig: 1 server seen: htdig: www.dlrg.de:80 2 documents htmerge: Total word count: 54429 htmerge: Total documents: 1 htmerge: Total doc db size (in K): 5

------------------------------------------------------------------------ Reiner Keller e-mail: DLRG@cs.TU-Berlin.de WWW : http://www.cs.TU-Berlin.de/~dlrg ------------------------------------------------------------------------ ---------------------------------------------------------------------- To unsubscribe from the htdig mailing list, send a message to htdig-request@sdsu.edu containing the single word "unsubscribe" in the body of the message.



This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:28:43 PST