Re: [htdig] ISO Info on Funtionality

Subject: Re: [htdig] ISO Info on Funtionality
From: Michael Johnson (
Date: Thu Dec 16 1999 - 14:21:42 PST

I was afraid of that. I will look into building a script that grabs a
page, reformats the links, and then displays it so that the domain does
not change. Nothing is ever easy is it. :)

Thanks for your help.

On Thu, 16 Dec 1999, Gilles Detillieux wrote:

> According to Michael Johnson:
> > I need a little information about how htdig handle header redirection.
> > Here is my situation. htdig is running on a domain and is limited to
> > searching only that domain. We want it to also index the first external
> > page from any level. Example:
> >
> > 1. is the site with htdig running
> >
> > 2. links to
> >
> > 3. In an attempt to make htdig index I
> > have done created a perl script that accepts 1 parameter, a url, and then
> > prints 'Location: [url]. This causes a redirection to [url].
> >
> > So my ultimate question is will htdig index this page or not? Does it see
> > that is a redirection and not follow the link?
> htdig will follow redirects, but only if they fall within the scope of
> limit_urls_to, which is set to the same as your start_url by default.
> You'd need to change this to allow digging files from other sites, but
> then it won't be limited to the first external page only.
> --
> Gilles R. Detillieux E-mail: <>
> Spinal Cord Research Centre WWW:
> Dept. Physiology, U. of Manitoba Phone: (204)789-3766
> Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930

SouthWind Internet Access, Inc. 120 S MarketSt. Suite 300
Michael Johnson (M.J.) Wichita, KS 67202
            Ignorance is not knowing that you don't know.

To unsubscribe from the htdig mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Thu Dec 16 1999 - 14:35:24 PST