[htdig] start_url and limit_url_to

Gabriel Fenteany (fenteany@calvin.bwh.harvard.edu)
Mon, 03 May 1999 21:22:18 -0400

Hi! I'd have the

start_url: http://foo.com/ http://foo2.com/ http://foo3.com/foofile.html
etc...

Some of the start_url URLs I was supplied by the site maintainers are in the
form "http://foo3.com/foofile.html" and they have no index file. Since I am
not in individual contact with all these people and am working off a URL
list, I can't get them all to rename "foofile.html" to "index.html"

However, I'd like to have all the <a href> linked local files on
foofile.html indexed, anyway.

The present limit_urls_to is like this

limit_urls_to: ${start_url}

Can I change the limit_urls_to another generic string that will get every
file on "http://foo3.com/" and sub-directories that is linked to
"http://foo3.com/foofile.html"?

(I know it would be better if they all had proper index files and I could
just use the URL http://fooX.com/ for each starting_url, but it might be
hard to contact all 300 sites and get them to all change them properly.)

Thanks very much.

Gabriel 

--
Gabriel Fenteany, Ph.D.
Post-doctoral Fellow
Tel: (617) 278-0390; Fax: (617) 734-2248
http://vl.bwh.harvard.edu
------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word "unsubscribe" in
the SUBJECT of the message.

This archive was generated by hypermail 2.0b3 on Mon May 03 1999 - 18:32:34 PDT