Re: [htdig] Identifying non-indexed URLs

Subject: Re: [htdig] Identifying non-indexed URLs
From: Gilles Detillieux (
Date: Tue Mar 14 2000 - 07:51:44 PST

According to Bigler, Tyson MT SSI:
> Is it possible to log URLs which are not indexed? The -vv flag will show
> level 1 & level 2 rejects due to explicit exceptions, but I'm interested in
> knowing which URLs were seen but not indexed because they weren't
> "parsable". Is this easily done?

In the 3.1.x series, the message is a bit misleading. With the -v flag,
htdig will report "not HTML" for any document it cannot parse. In 3.2.x,
this message is changed to the more general "not Parsable".

Gilles R. Detillieux              E-mail: <>
Spinal Cord Research Centre       WWW:
Dept. Physiology, U. of Manitoba  Phone:  (204)789-3766
Winnipeg, MB  R3E 3J7  (Canada)   Fax:    (204)789-3930

------------------------------------ To unsubscribe from the htdig mailing list, send a message to You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Tue Mar 14 2000 - 07:57:10 PST