Re: [htdig] Indexing large amount of non-related files

Subject: Re: [htdig] Indexing large amount of non-related files
From: Geoff Hutchison (
Date: Wed May 24 2000 - 06:06:51 PDT

At 12:54 PM +0200 5/24/00, Marcel Hicking wrote:
>I have been doing this for a much smaller site:
>I have set up a little shell script to generate
>a list with all available files and send it through
>sed to convert local paths to http://...-URLs.
>ht://dig is set up with start_url=allfiles.list
>and a local_urls line to "undo" the above mapping
>Do you think this is appropriate for a larger search
>or do you have any other suggestions?

For now this is probably fine. The 3.2 code has a file:// access
method, which would be idea for this purpose. One missing feature--it
doesn't auto-generate directory listings on-the-fly. Of course if
someone would be willing to write this, then indexing collections of
local files would be very easy.

-Geoff Hutchison
Williams Students Online

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Wed May 24 2000 - 04:12:56 PDT