Re: [htdig] URLs in url_list but not in DB


Geoff Hutchison (ghutchis@wso.williams.edu)
Tue, 3 Aug 1999 22:33:44 -0400 (EDT)


On Tue, 3 Aug 1999, Leonard J. Hunt wrote:

> http://www.foo.com/cgi-bin/application?14@@.ee6ebd3

That's truly a weird URL. In particular, there's no field separator in the
QUERY_STRING. To each his own, I guess.

> These URLs (or some of them) appear in the url_list after I
> run rundig, but they don't appear in the document database

The url_list file includes all URLs seen by htdig. This includes invalid
URLs.

How is exclude_urls set in your config file? The default excludes CGI
URLs.

-Geoff Hutchison
Williams Students Online
http://wso.williams.edu/

------------------------------------
To unsubscribe from the htdig mailing list, send a message to
htdig@htdig.org containing the single word unsubscribe in
the SUBJECT of the message.



This archive was generated by hypermail 2.0b3 on Tue Aug 03 1999 - 19:34:24 PDT