Re: [htdig] local_user_urls not working


Subject: Re: [htdig] local_user_urls not working
From: Gilles Detillieux (grdetil@scrc.umanitoba.ca)
Date: Thu May 25 2000 - 13:44:35 PDT


According to brian.chiang@philips.com:
> I installed the htdig on a Red Hat Linux 6.1 system. Basically the
> htdig is working fine with the Apache server, but the local_user_urls
> setting never works. As the attrs.html suggests, I have the following
> directive specified in my
> /etc/htdig/htdig.conf:
> local_user_urls: http://host.mydomain/=/home/,/public_html/
> I ran rundig several times but those files under /home/user/public_html
> never got indexed. It seems that htdig just skipped that part since
> I could not find any thing related from the script of "rundig -vvv".

All local_user_urls does is to tell htdig how to get files for URLs like
http://www.mydomain/~user/, if htdig encounters some of these URLs. It
does not tell htdig to read the /home directory to go looking for web
pages that are not linked to URLs in your start_urls attribute. htdig
is a spider - it follows HTML links from one page to another, and it
never reads directories on its own. If you want to index all the users'
home pages, you need to create a list of their URLs and feed it to htdig.

-- 
Gilles R. Detillieux              E-mail: <grdetil@scrc.umanitoba.ca>
Spinal Cord Research Centre       WWW:    http://www.scrc.umanitoba.ca/~grdetil
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to htdig-unsubscribe@htdig.org You will receive a message to confirm this.



This archive was generated by hypermail 2b28 : Thu May 25 2000 - 11:33:28 PDT