From: Stefan Sperling <stsp@stsp.name>
Subject: Re: Can got-index-pack handle >4G pack files?
To: Christian Weisgerber <naddy@mips.inka.de>
Cc: gameoftrees@openbsd.org
Date: Mon, 7 Feb 2022 01:16:07 +0100

On Sun, Feb 06, 2022 at 10:06:24PM +0100, Christian Weisgerber wrote:
> My attempt to test the "reuse deltas while packing" diff isn't doing
> so well.
> 
> ...
> packing 3437 references; 4069109 objects; deltify: 100%; writing pack:    4.7G 100%
> Wrote 0912cdde04551aaa49f0a8445912cdd03073c75f.pack
>    4.7G packed; indexing 11%  
> 
> got-index-pack has now accumulated over two hours of CPU time and
> is showing no signs of making progress.  I wonder if it's stuck in
> an infinite loop and if so, whether the pack file >4G is the problem.

This looks exactly like an issue which semarie@ has reported to me.
There is a bug where the deflate read code goes into an infinite loop.
If you attach with gdb you will likely see it looping there.
It happens if libz returns -5. The looping function does not handle
this case correctly. It does not seem to be an issue introduced by the
delta-reuse patch, but the patch exposes it.
I have not investigated in detail yet.