Re: [htdig] CGI Question

Subject: Re: [htdig] CGI Question
From: Gilles Detillieux (
Date: Wed Apr 05 2000 - 12:38:27 PDT

According to Tod Thomas:
> I commented out the exclude_urls line. The only other limiting configuration
> option I have in place is limit_urls_to which I have valued with our domain
> name. Still no luck. We do use cookies on this particular site but not to
> maintain session, only to provide user customization. Should no customization
> cookies exist, the initial page is a default so I would think that if anything
> htdig would have indexed the links presented on that default page.
> This particular site is driven by a central CGI process that dynamically
> generates all content. Using cookies a user has the ability to customize the
> presentation, but if they choose not to a default one is provided. My thought
> was htdig would have a problem indexing the site in its entirety since it
> would need to know what everybody's customizations were. I did think that
> htdig could index the default presentation though since cookies aren't
> required.
> Any thoughts?

All I can suggest at this point is to try htdig -vv to index your site,
and sift throught the output to see why it's rejecting URLs that you feel
it shouldn't - if it is indeed seeing them. Without that, we're all taking
stabs in the dark.

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Wed Apr 05 2000 - 11:37:49 PDT