Re: [htdig] indexing files generated at runtime

Subject: Re: [htdig] indexing files generated at runtime
From: Gilles Detillieux (
Date: Thu Dec 16 1999 - 14:17:03 PST

According to Jens Moellenhoff:
> We have some problems with the indexing of html files that are generated
> at runtime.
> >From a webserver (www.server), links go to a data bank server
> (www.databaseserver.ora) which itself gives out the result via the
> following link as html files:
> http://www.databaseserver.ora:1234/XXX/aaa/anzeige.produktlisten?sprach_id=de&produkt_art=Lebensmittel).
> The server www.server is fully indexed, but not the HTML file as it
> returns the result of the query from the database server.
> I added the data bank server to the htdig config file:
> limit_urls_to: http://www.server \
> http://databaseserver.ora:1234
> Why doesn't htdig index these pages? Can't htdig index files that are
> generated at runtime at all?

It should be able to do this, and I can't see any reason from what you've
said as to why it's not working for you, unless you're running a buggy old
3.1.0b2 beta release.

You should run htdig -vvv to see why it's rejecting the pages you want.

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Thu Dec 16 1999 - 14:30:50 PST