On Tue, Apr 15, 2025 at 08:11:42AM +0200, op@omarpolo.com wrote:
> Hello,
> 
> Colin reported on the IRC channel an absurd CPU consumption from
> got-fetch-http while cloning the freebsd src.git repository.  This boils
> down to some exaggerate memmove() in the http Transfer-Encoding: chunked
> parsing bits.
> 
> Diff below fixes it by avoiding these altogether during the http
> response handling, and instead using a cursor inside the read buffer.
> To be fair, I'm not entirely liking the diff, it feels more complicate
> than it really should be, but for now that's what i got.
> 
> While here I've thrown in it a small extra optimization: http_read() now
> knows how to directly write the data to a FILE in addition to a memory
> buffer.  This avoids an extra copy during the body handling since except
> for some stuff at the start, we just copy the body as-is to standard
> output.
> 
> With this, the majority of the time spent during the cloning of the
> freebsd src.git repository over http moves back to the delta resolving :)
> 
> Thoughts, comments or even oks?

> +
> +		if (out != NULL) {
> +			fwrite(rbuf->buf + rbuf->cur, 1, avail, out);

Please check for errors from fwrite() instead of ignoring them.
I realize this is always going to stdout, but I would prefer to keep
error checking idioms in place everywhere.