RE: htdig: won't index a group of pages

Ashley Hindmarsh (
Mon, 8 Jun 1998 09:34:38 +-100

Indexing from cgi isn't so bad - I've used htdig to index an on-line
database. The cgi script pumps out pages of links to all the records in
the database, which in turn builds HTML pages containing the textual data
with appropriate markups.

It's a 'paternally challenged' hack but since we had htdig on the database
server (Postgres / Linux) it was a quick and effective solution. If anyone
wants any more details on this, I'd be happy to oblige.


From: Michael Graff[]
Sent: 08 June 1998 09:16
To: Jed Michnowicz
Subject: Re: htdig: won't index a group of pages

Jed Michnowicz <> writes:

> "home.cgi?name=status"
> "home.cgi?name=contacts"
> etc...
> Htdig will index the page "home.cgi", but it will not index the links in
> the page that have the "home.cgi?name=somthing".

IMHO, "good" -- indexing generated pages from CGIs seems wrong. :)

> In the config file, I have limit_urls_to setup as:
> servername\cgi-bin\majordomo
> The document resides at:
> servername\cgi-bin\majordomo\home.cgi
> so this should not be the problem.

Complete with backslashes? Or forward, like they should be?


To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:32 PST