From: Benjamin Stürz <benni+got@stuerz.xyz>
Subject: Re: got-archive(1)
To: Stefan Sperling <stsp@stsp.name>
Cc: gameoftrees@openbsd.org
Date: Thu, 28 Dec 2023 12:51:51 +0100

On 12/28/23 10:44, Stefan Sperling wrote> Your tool takes a simple 
approach and works fine as a wrapper
> script. Still, it makes me wonder whether it would make sense
> to integrate something like it into Got itself.
That would be great.
I'll take a look into the code of got, and see if I can do something,
if no one is already working on it.
> My hope would be that having built-in tooling for th!is use case
> will encourage software maintainers to publish stable release
> artifacts instead of relying on auto-generated release artifacts
> such as those generated by Github.
> Github's "release" artifacts are generated on the fly with
> git-archive and cached for some time, but they will inevitably
> change over time because git-archive's output for a given input
> will eventually change after version upgrades. For example when
> the behaviour of file compression tooling changes. This has been
> an occasional pain to deal with in the OpenBSD ports tree.
> 
> In order to provide stable release tarballs for Got itself, we
> currently use a Makefile target which performs the following steps:
> 
>   1. Run 'make clean' to ensure that no build artifacts are packaged.
>   2. Ensure that the "-current" version number suffix gets dropped
>      from the compiled-in version string.
>   3. Build a list of files to package, and compare that to a known
>      list of files. In case of any difference in packaged files,
>      display the differences.
>   4. Generate the archive.
> 
> This approach has been working for Got, and I have successfully adapted
> this Makefile target for the NSH project. But there are some issues:
> 
> The approach is hidden in Got's Makefile and is not easy for users
> to discover. As such, we leave it up to users (like yourself) to
> devise some solution on their own. This in turn might push people
> towards relying on auto-generating services such as Github's
> release artifacts.
> 
> If the packaging list has changed, a useless tarball is left
> behind in the work tree, and the file got-version.mk is left in
> a modified state. Cleaning up the work tree before fixing things
> and retrying is a manual step. With integration into Got this
> could be handled in a more streamlined fashion.
> 
> Syncing the list of known files is a manual step and involves a
> temporary copy of the file. A built-in command would be able to
> display the diff and allow for packaging changes to be committed
> or rejected automatically in a manner similar to 'got stage -p'
> followed by 'got commit'.
> 
> Tools like autoconf produce unversioned files to be included in the
> tarball and require a more complicated approach than your script
> provides.
My script was intended to be run on the gotd's server,
by some script invoked by a cronjob.
See my update-gotd script below.
> Managing a list of files to package becomes unavoidable if
> such projects would want to make use of this feature. One such project
> is got-portable.
> 
When I used autoconf for my project (https://github.com/riscygeek/bcc),
I'd just run ./autogen.sh and check everything in, before releasing
a new version. After which I ran make full-clean,
a custom rule that removes everything configure-related.
After that I'd create a release on GitHub and use their tarballs.
In retrospect, I should have created a new branch for every release,
and nowadays I'd use my own infrastructure,
but the same principle would still apply,
if I used autoconf (which I thankfully don't).
I don't know if this is the right way to do so, but it worked for me.


/usr/local/sbin/update-gotd:
> #!/bin/sh
> 
> gen() {
>         cat /var/git/conf/head
>         for repo in /var/git/*.git; do
>                 name=$(basename "$repo" .git)
>                 owner=$(stat -f '%Su' "$repo/owner")
>                 sed                             \
>                         -e "s/@NAME@/$name/"    \
>                         -e "s#@PATH@#$repo#"    \
>                         -e "s/@OWNER@/$owner/"  \
>                         < /var/git/conf/repo
>         done
> }
> 
> /usr/bin/openrsync --rsync-path=/usr/bin/openrsync -rt --delete /var/git/*.git /var/www/got/public/
> 
> tmpfile=$(mktemp)
> gen > "$tmpfile"
> chmod 644 "$tmpfile"
> 
> for repo in /var/git/*.git; do
>         got-archive -aso "/var/www/htdocs/got.stuerz.xyz/download" "$repo"
> done
> 
> if ! cmp -s /etc/gotd.conf "$tmpfile"; then
>         echo "Updatd gotd.conf:" >&2
>         diff -u /etc/gotd.conf "$tmpfile" >&2
> 
>         mv "$tmpfile" /etc/gotd.conf
>         rcctl restart gotd
> fi
> 
> exit 0
This script is run every minute,
ideally I'd only want it to be run when I do `got send`,
but I haven't found a way to do it.
Maybe make a custom got-receive-pack/got-upload-pack?

For context, my /usr/local/sbin/newrepo:
> #!/bin/sh
> 
> [ $# -ne 3 ] && { echo "Usage: $0 name owner description"; exit 1; }
> 
> name=$1
> owner=$2
> descr=$3
> 
> path="/var/git/${name}.git"
> owner_name=$(userinfo "${owner}" | grep '^gecos' | cut -f2)
> 
> gotadmin init "${path}"
> chown -R _gotd "${path}"
> printf '%s\n' "$descr" > "${path}/description"
> printf '\n[gotweb]\n\towner = %s\n' "${owner_name}" >> "${path}/config"
> printf 'ssh://anon@stuerz.xyz/%s.git\n' "${name}" > "${path}/cloneurl"
> touch "${path}/owner"
> chown "${owner}" "${path}/owner"
> chmod 444 "${path}/owner"
> update-gotd