Re: [htdig3-dev] size of dynamic pages

Subject: Re: [htdig3-dev] size of dynamic pages
From: Gabriele Bartolini (
Date: Wed Feb 09 2000 - 03:04:49 PST

Ciao talponi, how are you doing?

>I did find a problem in ReadChunkedBody(). It appends to the _contents
>string using "<<", rather than using
> _response._contents.append(buffer, rsize);
>as ReadBody() does. The "<<" will fail if there are any null characters
>in the buffer, which may not be a problem for text/* content types,
>but will be for binary files.

Fixed ;-)

>Also, ReadBody() and ReadChunkedBody() are inconsistent in the way they
>set _response._content_length & _response._document_length. ReadBody()
>doesn't even make sure _response._content_length gets set. They should
>both end with something like:
> _response._document_length = _response._contents.length();

You mean, maybe:

> if (_response._content_length < _response._document_length)
> _response._content_length = _response._document_length;

And why?

Well, as far as ReadChunkedBody() is, there's another problem. We don't use
max_document_size attribute, and I have no idea on how to use it (but
closing the connection as the size has been reached -- in this way I'll
never know how much the content-length is). That sucks ;-(

Any ideas?

>Either that, or this should be done once in HtHTTP::HTTPRequest(),
>after one or the other of the Read*Body() methods is called. I'm not
>completely clear on the chunked v. unchunked reading, but it seems to me
>this should not affect how the content length and type are processed.
>If it's guaranteed that when the body is chunked, there will be no
>Content-Length header, then I guess the comparison between _content_length
>and _document_length is not needed, and you can just set _content_length

I found the answer ... and I hope I'll get to be clear in the explanation.

If you do 'telnet localhost 80' and type this (in order to ask for a normal
HTML page)

host: localhost

I received:

HTTP/1.1 200 OK
Date: Wed, 09 Feb 2000 10:37:43 GMT
Server: Apache/1.3.9 (Unix) PHP/3.0.11
Last-Modified: Fri, 04 Feb 2000 07:53:39 GMT
ETag: "3e93b-9c5-389a8583"
Accept-Ranges: bytes
Content-Length: 2501
Content-Type: text/html

In the other case, if you ask for a dynamic page, e.g. like a PHP one, by
doing 'telnet localhost 80', then:

GET /prove/prova.php3 HTTP/1.1
host: localhost

I receive this:

HTTP/1.1 200 OK
Date: Wed, 09 Feb 2000 10:40:15 GMT
Server: Apache/1.3.9 (Unix) PHP/3.0.11
Transfer-Encoding: chunked
Content-Type: text/html

<TITLE>Pagina di prova del PHP</TITLE>

<BR>Nome: <BR>


As you can see, if Transfer-Encoding is set to chunked, no Content-Length
is given (how can the server send it if it doesn't know it).

Besides, if you look at HTTP1/1's RFC, chapter 4.4 "Message Lenght", point
2, you can read:

<< If a Transfer-Encoding header field is present and has any value other
than "identity", then the transfer-length is defined by use of the
"chunked" transfer-encoding, unless the message is terminated by closing
the connection. >>

And in this case, how can I determine the content-length?


<< All HTTP/1.1 applications that receive entities MUST accept the
"chunked" transfer-coding, thus allowing this mechanism to be used for
messages when the message length cannot be determined in advance. >>

Is htdig an HTTP/1.1 applications? I suppose yes ...

And finally let's get to the point:

<< Messages MUST NOT include both a Content-Length header field and a
non-identity transfer-coding. If the message does include a non-identity
transfer coding, the Content-Length MUST be ignored. >>

Any assesments and considerations?

I hope this can be useful to all of you ...


P.S.: There's an obscure side regarding Chunks, and concerns the Trailer of
the message. Maybe the server can set some header fields here. I remember I
did some attempts and I always got an empty trailer ... Probably you know
better than me and you can help me.


Gabriele Bartolini
Computer Programmer (are U sure?)
U.O. Rete Civica - Comune di Prato
Prato - Italia - Europa


Zinedine "Zizou" Zidane. Just for soccer lovers.

To unsubscribe from the htdig3-dev mailing list, send a message to
You will receive a message to confirm this.

This archive was generated by hypermail 2b28 : Wed Feb 09 2000 - 03:08:31 PST