Re: [htdig] External Converter Prob

Subject: Re: [htdig] External Converter Prob
From: Gilles Detillieux (
Date: Thu Jan 11 2001 - 07:41:36 PST

According to Reich, Stefan:
> all my descriptions are starting with "content-type: text/html".
> Is this normal behavior or is it, because I'm using an external converter to
> do some modifications on the spidered html files. I registered my converter
> for text/html -> text/myhtml conversion. I've patched the html parser to
> recognize this in addition to text/html.
> I'm sure my external converter doesn't write text/html to the output stream.
> Any ideas?

No, this is not normal behaviour. If you're certain that your external
converter doesn't write this out, then we'd have to assume it comes
from elsewhere. It may be a stupid question, but are you sure the pages
you're indexing don't contain this extra header? I've seen defective
CGI scripts, for example, that inadvertantly output two such headers in
some situations. Ditto for SSI pages that call CGI scripts incorrectly.
Finally, it's hard to be sure it isn't a problem with your patches
to htdig, or to your particular configuration, without being able to
see them. I don't know if this helps or not, but it may give you a few
more places to look.

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this. List archives: <> FAQ: <>

This archive was generated by hypermail 2b28 : Thu Jan 11 2001 - 07:55:27 PST