Re: htdig: indexing pages without links

Geoff Hutchison (
Wed, 30 Sep 1998 11:35:58 -0400

>I mean I've got a lot of individual pages in my directories but no page
>have links to these pages. How can I index (dig ?) these pages ? I know
>it's possible with EWS because EWS works with local pages and
>directories and reads the whole directory content. So even if there's no
>link to some pages they are indexed as well as linked ones.

I've discussed this with Andrew Bishop, the author of WWWoffle (a local
HTTP cache). The easiest way to do this is to write a small script that
generates a list of URLs into a document for the start_urls directive.

e.g. (in /bin/sh script)

for file in `ls dir1/* dir2/*`; do
  echo$file >>start_url_file

This will produce a file of URLs:

>I've defined local_urls but it doesn't do what I want.

You'll need this defined to translate the URLs into files.

-Geoff Hutchison
Williams Students Online

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:27:54 PST