From: Stefan Sperling Subject: Re: got clone: start of effort. To: Ori Bernstein , gameoftrees@openbsd.org Date: Wed, 18 Mar 2020 17:07:59 +0100 On Fri, Feb 21, 2020 at 09:58:14AM +0100, Stefan Sperling wrote: > On Mon, Feb 17, 2020 at 02:13:30PM -0500, Ori Bernstein wrote: > > Now tested against a clean clone. > > Thanks, it applied and compiled fine. > > I have started making changes on top of this diff. Because there seems > to be a lot of work left to do before this can be released I am working > on a branch for now: > https://git.gameoftrees.org/gitweb/?p=got.git;a=shortlog;h=refs/heads/clone > > Everyone is of course welcome to contribute there. > > Thanks a lot for getting this started Ori! At this point I think enough progress has been made on this branch for integration. The git:// protocol works, as does ssh:// (tested against github). Currently only full clones are supported, a 'got fetch' command for incremental updates will come later. The new helpers 'got-fetch-pack' and 'got-index-pack' are functional and pledged with "stdio recvfd". The main 'got clone' program uses the usual pledges plus "dns inet" if the git:// protocol is used (otherwise ssh(1) handles those things). got-index-pack is single-threaded and its performance is notably worse than 'git index-pack' which runs one thread per CPU core. This can be improved later if needed. Cloning src.git over localhost on a Matebook X takes between 6 and 7 minutes, and requires about 250 MB of RAM. About 50 MB of that are used to store the generated pack index and could be spilled into a temporary file but I doubt that will help much on memory constrained machines. I don't remember how much time a full src CVS reposync from scratch takes on a fast link. Does anyone know? Memory usage will definitely be lower with rsync. In any case, it would be possible to use rsync for initial cloning of Git repositories and the Git protocol for incremental updates. I will now look at rebasing the branch and squashing the history a bit before integrating it. It does not look like anyone else has been actively working on the branch so I'm not going to wait and just do it.