Re: [htdig] local_user_urls: query

Subject: Re: [htdig] local_user_urls: query
Date: Thu Feb 10 2000 - 01:10:58 PST

> According to Geoff Hutchison:
> > At 9:09 AM +0000 2/9/00, wrote:
> > >Does htdig recognize that some files may have server-side includes and
> > >always fetch them via http despite these attributes in the config file?
> > >
> > >An Apache server will process SSIs in .shtm and .shtml files, plus .htm
> > >and .html files with execution permission set.
> >
> > Currently, the local_* attributes only read .htm and .html files. It
> > makes no attempt to emulate server-parsing. So if you have set
> > XBitHack for your Apache server, there isn't any way htdig will know
> > that and it will fly right through, ignoring your SSI code.
> >
> > However, .shtml, .phtml, .php3 files and the like will not be indexed
> > through the local filesystem, instead going to HTTP.
> Actually, htdig 3.1.4 also accepts .txt, .asc, .pdf, .ps and .eps files
> locally. For some reason, that change never made it into 3.2.0b1.
> I imagine it got lost in the merge. Anyway, with 3.2's mime.types
> support, that's the way RetrieveLocal() should determine the content-type
> for local files. It'll just need a few lines of code to add that in,
> I expect.
> In any case, htdig has no equivalent to Apache's XBitHack, so for SSI
> documents, I'd recommend using .shtml if you want server-side parsing.
> For my own system, I use SSI only to add a few bits and pieces, so I
> don't mind that that stuff doesn't get indexed. I now index everything
> through local_urls.

I must ask: why does htdig handle local_urls this way?

Doesn't it make more sense, when the attribute is set, to get ALL pages
locally except when:

        the file does not exist
        the file has execution permission set
        the file name ends in .shtm, .shtml and one or two others
        which have special meanings to servers.

> --
> Gilles R. Detillieux E-mail: <>
> Spinal Cord Research Centre WWW:
> Dept. Physiology, U. of Manitoba Phone: (204)789-3766
> Winnipeg, MB R3E 3J7 (Canada) Fax: (204)789-3930

David J Adams
Computing Services
University of Southampton

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Thu Feb 10 2000 - 01:13:44 PST