[htdig] why doesnt htdig search all directories?

Sadhunathan Nadesan (sadhu@aloha.net)
Wed, 15 Sep 1999 06:19:53 -0700


sending in this plea again. thank you for the suggestions of, removing old
data ase, starting over. it didnt help. can anyone tell me please, what
causes htdig to not index everything?

i have my start url listed as:

start_url: http://www.gurudeva.dynip.com/~htoday http://www.hinduismtoday
kauai.hi.us http://www.hindu.org

just a cgi bin directory in the exclude urls, limit to set to equal start
url, and,
in the log file after reindexing i can clearly see, htdig is not traversing
every directory under these start urls .. just some of them.

what causes it to do some, and skip others?

could it have anything to do with 'not found' errors? is there a limit?


To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Wed Sep 15 1999 - 06:35:36 PDT