From: ori@eigenstate.org
Subject: Re: got clone: start of effort.
To: stsp@stsp.name, ori@eigenstate.org, gameoftrees@openbsd.org
Date: Thu, 19 Mar 2020 20:08:06 -0700

> On Fri, Feb 21, 2020 at 09:58:14AM +0100, Stefan Sperling wrote:
>> On Mon, Feb 17, 2020 at 02:13:30PM -0500, Ori Bernstein wrote:
>> > Now tested against a clean clone.
>> 
>> Thanks, it applied and compiled fine.
>> 
>> I have started making changes on top of this diff. Because there seems
>> to be a lot of work left to do before this can be released I am working
>> on a branch for now:
>> https://git.gameoftrees.org/gitweb/?p=got.git;a=shortlog;h=refs/heads/clone
>> 
>> Everyone is of course welcome to contribute there.
>> 
>> Thanks a lot for getting this started Ori!
> 
> At this point I think enough progress has been made on this branch
> for integration.
> 
> The git:// protocol works, as does ssh:// (tested against github).
> Currently only full clones are supported, a 'got fetch' command for
> incremental updates will come later.
> 
> The new helpers 'got-fetch-pack' and 'got-index-pack' are functional and
> pledged with "stdio recvfd". The main 'got clone' program uses the usual
> pledges plus "dns inet" if the git:// protocol is used (otherwise ssh(1)
> handles those things).
> 
> got-index-pack is single-threaded and its performance is notably worse
> than 'git index-pack' which runs one thread per CPU core. This can be
> improved later if needed. Cloning src.git over localhost on a Matebook X
> takes between 6 and 7 minutes, and requires about 250 MB of RAM. About
> 50 MB of that are used to store the generated pack index and could be
> spilled into a temporary file but I doubt that will help much on memory
> constrained machines.
> 
> I don't remember how much time a full src CVS reposync from scratch takes
> on a fast link. Does anyone know? Memory usage will definitely be lower
> with rsync. In any case, it would be possible to use rsync for initial
> cloning of Git repositories and the Git protocol for incremental updates.
> 
> I will now look at rebasing the branch and squashing the history a bit
> before integrating it. It does not look like anyone else has been actively
> working on the branch so I'm not going to wait and just do it.

Great -- at some point in the next few weeks, I'll look at
git push; I don't think we have redeltification yet, so I'll
just do trivial packfiles, with no deltas at all.  We can
revisit that later fairly easily.