On Fri, Feb 21, 2020 at 09:58:14AM +0100, Stefan Sperling wrote:
> On Mon, Feb 17, 2020 at 02:13:30PM -0500, Ori Bernstein wrote:
> > Now tested against a clean clone.
> 
> Thanks, it applied and compiled fine.
> 
> I have started making changes on top of this diff. Because there seems
> to be a lot of work left to do before this can be released I am working
> on a branch for now:
> https://git.gameoftrees.org/gitweb/?p=got.git;a=shortlog;h=refs/heads/clone
> 
> Everyone is of course welcome to contribute there.
> 
> Thanks a lot for getting this started Ori!

At this point I think enough progress has been made on this branch
for integration.

The git:// protocol works, as does ssh:// (tested against github).
Currently only full clones are supported, a 'got fetch' command for
incremental updates will come later.

The new helpers 'got-fetch-pack' and 'got-index-pack' are functional and
pledged with "stdio recvfd". The main 'got clone' program uses the usual
pledges plus "dns inet" if the git:// protocol is used (otherwise ssh(1)
handles those things).

got-index-pack is single-threaded and its performance is notably worse
than 'git index-pack' which runs one thread per CPU core. This can be
improved later if needed. Cloning src.git over localhost on a Matebook X
takes between 6 and 7 minutes, and requires about 250 MB of RAM. About
50 MB of that are used to store the generated pack index and could be
spilled into a temporary file but I doubt that will help much on memory
constrained machines.

I don't remember how much time a full src CVS reposync from scratch takes
on a fast link. Does anyone know? Memory usage will definitely be lower
with rsync. In any case, it would be possible to use rsync for initial
cloning of Git repositories and the Git protocol for incremental updates.

I will now look at rebasing the branch and squashing the history a bit
before integrating it. It does not look like anyone else has been actively
working on the branch so I'm not going to wait and just do it.