Re: htdig: looping with lots of / in URL

John Lines (
Mon, 13 Jul 1998 10:25:35 +0100

> Hello!
> I found interesting bug in 3.08b2 which is nice way for denial of
> service. This is from web server's logfile:
> - - [11/Jul/1998:20:30:43 +0200] "GET
> /si//////si/ris98.html HTT
> P/1.0" 404 174 "" "htdig/3.0.8b2
> (andrew@
> - - [11/Jul/1998:20:30:44 +0200] "GET
> /si//////si/ris98.html/ HT
> TP/1.0" 404 175 "" "htdig/3.0.8b2
> (andrew
> See lots of / in path? They just keep growing and filling things. Looks
> like some bad url in html
> made htdig loop.

We had the same problem some time ago.

> Was this fixed in some patch already?

The problem is that strictly speaking
is different from, i.e. they could
legally point to different pieces of information within the HTTP specification.

On most Unix, Netware or Microsoft servers they will return the same info,
but they dont have to, so htdig is doing the correct thing.

We solved ours by an exclude_urls directive in htdig.conf, and then tracking
down the bad bit of HTML.

I suppose htdig could get the server type when it talks to a new server and
set a flag if it recognises the server and OS in which it would collapse
multiple //, and possibly handle collapsing to lower case etc - depending
on the server OS - but this could be quite a bit of work, particularly in the
testing to make sure it didnt break anything.

> Thanks in advance.
> Tomaz
> p.s.
> Thanks to Andrew for nice software.


                John Lines

To unsubscribe from the htdig mailing list, send a message to containing the single word "unsubscribe" in
the body of the message.

This archive was generated by hypermail 2.0b3 on Sat Jan 02 1999 - 16:26:52 PST