[htdig3-dev] Problem: same urls in doc database

Subject: [htdig3-dev] Problem: same urls in doc database
From: Valdas Andrulis (valdand@soften.ktu.lt)
Date: Mon Feb 21 2000 - 07:12:49 PST


I am testing htdig-3.2.0 beta.

Here it is my setup
persistent_connections: Yes
head_before_get: Yes
max_connection_requests: 30

compression_level: 6
wordlist_compress: true
wordlist_compress_debug: 0
wordlist_cache_size: 20000000
wordlist_monitor: false

locale: lt_LT
excerpt_length: 300

also i have 2 servers, one have about 50 documents, all are static, and
one server has only one document, which is dynamic.

I run thease commands:
htdig -v -i
htmerge -v -s

and repeatedly:
htdig -v
htmerge -v -s

Af course dynamic page is allways changed. and after each second run
document count increases by one, i.e. at first there are one
http://anotherserver/ url, then two urls, three urls, etc.

I think there something with last record in database. When pushing urls
already in database each second run it dosn't push url
http://anotherserver/ from database, but insted from start_urls, i.e.
that means that thereis no such url in database! This is weired!?

Can you check this behavior?



To unsubscribe from the htdig3-dev mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Mon Feb 21 2000 - 07:16:19 PST