Erick Thompson (firstname.lastname@example.org)
Mon, 27 Sep 1999 20:43:27 -0700
Thanks for the info. However, if there is a link from the subdirectory to
the root url (foo.com in this case) will htdig follow that link? In the
config file, I have the restrict URL set to the start URL(s). Is this
restriction based on the address or the url? That is, if the start URL is
www.foo.com/data/, will www.foo.com pass the restrict URL test?
At 08:21 PM 9/27/99 -0600, you wrote:
>Erick Thompson's bits of Mon, 27 Sep 1999 translated to:
> > I've just installed htdig, and I was wondering how it handles a directory
> > name as part of the base URL. That is, if I put in www.foo.com/data/ will
> > htdig index all of www.foo.com or just the documents under data?
>It should do neither. Instead, it will look for a default page in the data/
>directory (e.g. index.html, index.htm). It will index that page, collect
>all of the links listed on that page, and proceed to do the same with each
>of those links. There are a number of ways in which the links it follows
>will be limited, but it will only consider pages that can be reached by
>traversing a series of links that originate in the default page.
To unsubscribe from the htdig mailing list, send a message to
email@example.com containing the single word unsubscribe in
the SUBJECT of the message.
This archive was generated by hypermail 2.0b3 on Mon Sep 27 1999 - 20:45:34 PDT