add 'got send' command

This patch implements 'got send', which uploads changes to a Git server. Tests would be appreciated. Beyond the regression tests added in the patch I have verified that a simple change can be sent to Github: https://github.com/stspdotname/send-test/commit/9a8faf08f14f0580ff0600a2dc82224e460441d3 Please try to spot issues in the documentation and code added here. I realize this is a lot to review. I don't expect anyone to check every single line of code. My progress here would be to add the feature to the repository once initial test results are good, and make a release. Then make another release a week or so later (during k2k21?) with fixes. Meanwhile, do not hesitate to keep reading code if you want to audit it. We can always fix and release quickly. Some code from got-fetch-pack is duplicated in got-send-pack. I intend to factor out common code into a new file in the lib/ directory later. But my main focus for now is to have this new feature working. diff d5f2edc6096c1698c1ec74b74ef1ec5a4b32783c 86e5d0bbf3b96cfda42f73cfca32bbe3539073f0 blob - 29c7f19e9b90a3fa15fae12f16e5e7654db9f42e blob + 14f769b331dbbbc7a4e4a948191670b97122457d --- got/Makefile +++ got/Makefile @@ -12,7 +12,7 @@ SRCS= got.c blame.c commit_graph.c delta.c diff.c \ gotconfig.c diff_main.c diff_atomize_text.c \ diff_myers.c diff_output.c diff_output_plain.c \ diff_output_unidiff.c diff_output_edscript.c \ - diff_patience.c + diff_patience.c send.c deltify.c pack_create.c MAN = ${PROG}.1 got-worktree.5 git-repository.5 got.conf.5 blob - 2fc788135c449d339f9e22c8ac5d28eaf3bb8047 blob + 107ed8f094e31e980483426f5aee481ff5d9935e --- got/got.1 +++ got/got.1 @@ -1372,6 +1372,185 @@ in the repository. .It Cm ci Short alias for .Cm commit . +.It Cm send Oo Fl a Oc Oo Fl b Ar branch Oc Oo Fl d Ar branch Oc Oo Fl f Oc Oo Fl r Ar repository-path Oc Oo Fl t Ar tag Oc Oo Fl T Oc Oo Fl q Oc Oo Fl v Oc Op Ar remote-repository +Send new changes to a remote repository. +If no +.Ar remote-repository +is specified, +.Dq origin +will be used. +The remote repository's URL is obtained from the corresponding entry in +.Xr got.conf 5 +or Git's +.Pa config +file of the local repository, as created by +.Cm got clone . +.Pp +All objects corresponding to new changes will be written to a temporary +pack file which is then uploaded to the server. +Upon success, references in the +.Dq refs/remotes/ +reference namespace of the local repository will be updated to point at +the commits which have been sent. +.Pp +By default, changes will only be sent if they are based on up-to-date +copies of relevant branches in the remote repository. +If any changes to be sent are based on out-of-date copies, new changes +must be fetched from the server with +.Cm got fetch +and local branches must be rebased with +.Cm got rebase +before +.Cm got send +can succeed. +.Pp +The options for +.Cm got send +are as follows: +.Bl -tag -width Ds +.It Fl a +Send all branches from the local repository's +.Dq refs/heads/ +reference namespace. +The +.Fl a +option is equivalent to listing all branches with multiple +.Fl b +options. +Cannot be used together with the +.Fl b +option. +.It Fl b Ar branch +Send the specified +.Ar branch +from the local repository's +.Dq refs/heads/ +reference namespace. +This option may be specified multiple times to build a list of branches +to send. +If this option is not specified, default to the work tree's current branch +if invoked in a work tree, or to the repository's HEAD reference. +Cannot be used together with the +.Fl a +option. +.It Fl d +Delete the specified +.Ar branch +from the remote repository's +.Dq refs/heads/ +reference namespace. +This option may be specified multiple times to build a list of branches +to delete. +.Pp +Only references are deleted. +Any commit, tree, tag, and blob objects belonging to deleted branches +may become subject to deletion by Git's garbage collector running on +the server. +.Pp +Requesting deletion of branches results in an error if the server +does not support this feature. +.It Fl f +Attempt to force the server to accept uploaded branches or tags in +spite of failing client-side sanity checks. +The server may reject forced requests regardless, depending on its +configuration. +.Pp +Any commit, tree, tag, and blob objects belonging to overwritten branches +or tags may become subject to deletion by Git's garbage collector running +on the server. +.Pp +The +.Dq refs/tags +reference namespace is globally shared between all repositories. +Use of the +.Fl f +option to overwrite tags is discouraged because it can lead to +inconsistencies between the tags present in different repositories. +In general, creating a new tag with a different name is recommended +instead of overwriting an existing tag. +.Pp +Use of the +.Fl f +option is particularly discouraged if changes being sent are based +on an out-of-date copy of a branch in the remote repository. +Instead of using the +.Fl f +option, new changes should +be fetched with +.Cm got fetch +and local branches should be rebased with +.Cm got rebase , +followed by another attempt to send the changes. +.Pp +The +.Fl f +option should only be needed in situations where the remote repository's +copy of a branch or tag is known to be out-of-date and is considered +disposable. +The risks of creating inconsistencies between different repositories +should also be taken into account. +.It Fl r Ar repository-path +Use the repository at the specified path. +If not specified, assume the repository is located at or above the current +working directory. +If this directory is a +.Nm +work tree, use the repository path associated with this work tree. +.It Fl t Ar tag +Send the specified +.Ar tag +from the local repository's +.Dq refs/tags/ +reference namespace, in addition to any branches that are being sent. +The +.Fl t +option may be specified multiple times to build a list of tags to send. +No tags will be sent if the +.Fl t +option is not used. +.Pp +Raise an error if the specified +.Ar tag +already exists in the remote repository, unless the +.Fl f +option is used to overwrite the sever's copy of the tag. +In general, creating a new tag with a different name is recommended +instead of overwriting an existing tag. +.Pp +Cannot be used together with the +.Fl T +option. +.It Fl T +Attempt to send all tags from the local repository's +.Dq refs/tags/ +reference namespace. +The +.Fl T +option is equivalent to listing all tags with multiple +.Fl t +options. +Cannot be used together with the +.Fl t +option. +.It Fl q +Suppress progress reporting output. +The same option will be passed to +.Xr ssh 1 +if applicable. +.It Fl v +Verbose mode. +Causes +.Cm got send +to print debugging messages to standard error output. +The same option will be passed to +.Xr ssh 1 +if applicable. +Multiple -v options increase the verbosity. +The maximum is 3. +.El +.It Cm se +Short alias for +.Cm send . .It Cm cherrypick Ar commit Merge changes from a single .Ar commit blob - 71cc16e4b52a05afc8991d3d0c566bd8c200b3d7 blob + 5f57676c71bfe6ae726a39f6cba1b9fb3dafb730 --- got/got.c +++ got/got.c @@ -51,6 +51,7 @@ #include "got_diff.h" #include "got_commit_graph.h" #include "got_fetch.h" +#include "got_send.h" #include "got_blame.h" #include "got_privsep.h" #include "got_opentemp.h" @@ -102,6 +103,7 @@ __dead static void usage_add(void); __dead static void usage_remove(void); __dead static void usage_revert(void); __dead static void usage_commit(void); +__dead static void usage_send(void); __dead static void usage_cherrypick(void); __dead static void usage_backout(void); __dead static void usage_rebase(void); @@ -130,6 +132,7 @@ static const struct got_error* cmd_add(int, char *[]) static const struct got_error* cmd_remove(int, char *[]); static const struct got_error* cmd_revert(int, char *[]); static const struct got_error* cmd_commit(int, char *[]); +static const struct got_error* cmd_send(int, char *[]); static const struct got_error* cmd_cherrypick(int, char *[]); static const struct got_error* cmd_backout(int, char *[]); static const struct got_error* cmd_rebase(int, char *[]); @@ -159,6 +162,7 @@ static struct got_cmd got_commands[] = { { "remove", cmd_remove, usage_remove, "rm" }, { "revert", cmd_revert, usage_revert, "rv" }, { "commit", cmd_commit, usage_commit, "ci" }, + { "send", cmd_send, usage_send, "se" }, { "cherrypick", cmd_cherrypick, usage_cherrypick, "cy" }, { "backout", cmd_backout, usage_backout, "bo" }, { "rebase", cmd_rebase, usage_rebase, "rb" }, @@ -7353,6 +7357,498 @@ done: } __dead static void +usage_send(void) +{ + fprintf(stderr, "usage: %s send [-a] [-b branch] [-d branch] [-f] " + "[-r repository-path] [-t tag] [-T] [-q] [-v] " + "[remote-repository]\n", getprogname()); + exit(1); +} + +struct got_send_progress_arg { + char last_scaled_packsize[FMT_SCALED_STRSIZE]; + int verbosity; + int last_ncommits; + int last_nobj_total; + int last_p_deltify; + int last_p_written; + int last_p_sent; + int printed_something; + int sent_something; + struct got_pathlist_head *delete_branches; +}; + +static const struct got_error * +send_progress(void *arg, off_t packfile_size, int ncommits, int nobj_total, + int nobj_deltify, int nobj_written, off_t bytes_sent, const char *refname, + int success) +{ + struct got_send_progress_arg *a = arg; + char scaled_packsize[FMT_SCALED_STRSIZE]; + char scaled_sent[FMT_SCALED_STRSIZE]; + int p_deltify = 0, p_written = 0, p_sent = 0; + int print_searching = 0, print_total = 0; + int print_deltify = 0, print_written = 0, print_sent = 0; + + if (a->verbosity < 0) + return NULL; + + if (refname) { + const char *status = success ? "accepted" : "rejected"; + + if (success) { + struct got_pathlist_entry *pe; + TAILQ_FOREACH(pe, a->delete_branches, entry) { + const char *branchname = pe->path; + if (got_path_cmp(branchname, refname, + strlen(branchname), strlen(refname)) == 0) { + status = "deleted"; + break; + } + } + } + + printf("\nServer has %s %s", status, refname); + a->printed_something = 1; + return NULL; + } + + if (fmt_scaled(packfile_size, scaled_packsize) == -1) + return got_error_from_errno("fmt_scaled"); + if (fmt_scaled(bytes_sent, scaled_sent) == -1) + return got_error_from_errno("fmt_scaled"); + + if (a->last_ncommits != ncommits) { + print_searching = 1; + a->last_ncommits = ncommits; + } + + if (a->last_nobj_total != nobj_total) { + print_searching = 1; + print_total = 1; + a->last_nobj_total = nobj_total; + } + + if (packfile_size > 0 && (a->last_scaled_packsize[0] == '\0' || + strcmp(scaled_packsize, a->last_scaled_packsize)) != 0) { + if (strlcpy(a->last_scaled_packsize, scaled_packsize, + FMT_SCALED_STRSIZE) >= FMT_SCALED_STRSIZE) + return got_error(GOT_ERR_NO_SPACE); + } + + if (nobj_deltify > 0 || nobj_written > 0) { + if (nobj_deltify > 0) { + p_deltify = (nobj_deltify * 100) / nobj_total; + if (p_deltify != a->last_p_deltify) { + a->last_p_deltify = p_deltify; + print_searching = 1; + print_total = 1; + print_deltify = 1; + } + } + if (nobj_written > 0) { + p_written = (nobj_written * 100) / nobj_total; + if (p_written != a->last_p_written) { + a->last_p_written = p_written; + print_searching = 1; + print_total = 1; + print_deltify = 1; + print_written = 1; + } + } + } + + if (bytes_sent > 0) { + p_sent = (bytes_sent * 100) / packfile_size; + if (p_sent != a->last_p_sent) { + a->last_p_sent = p_sent; + print_searching = 1; + print_total = 1; + print_deltify = 1; + print_written = 1; + print_sent = 1; + } + a->sent_something = 1; + } + + if (print_searching || print_total || print_deltify || print_written || + print_sent) + printf("\r"); + if (print_searching) + printf("packing %d reference%s", ncommits, + ncommits == 1 ? "" : "s"); + if (print_total) + printf("; %d object%s", nobj_total, + nobj_total == 1 ? "" : "s"); + if (print_deltify) + printf("; deltify: %d%%", p_deltify); + if (print_sent) + printf("; uploading pack: %*s %d%%", FMT_SCALED_STRSIZE, + scaled_packsize, p_sent); + else if (print_written) + printf("; writing pack: %*s %d%%", FMT_SCALED_STRSIZE, + scaled_packsize, p_written); + if (print_searching || print_total || print_deltify || + print_written || print_sent) { + a->printed_something = 1; + fflush(stdout); + } + return NULL; +} + +static const struct got_error * +cmd_send(int argc, char *argv[]) +{ + const struct got_error *error = NULL; + char *cwd = NULL, *repo_path = NULL; + const char *remote_name; + char *proto = NULL, *host = NULL, *port = NULL; + char *repo_name = NULL, *server_path = NULL; + const struct got_remote_repo *remotes, *remote = NULL; + int nremotes, nbranches = 0, ntags = 0, ndelete_branches = 0; + struct got_repository *repo = NULL; + struct got_worktree *worktree = NULL; + const struct got_gotconfig *repo_conf = NULL, *worktree_conf = NULL; + struct got_pathlist_head branches; + struct got_pathlist_head tags; + struct got_reflist_head all_branches; + struct got_reflist_head all_tags; + struct got_pathlist_head delete_args; + struct got_pathlist_head delete_branches; + struct got_reflist_entry *re; + struct got_pathlist_entry *pe; + int i, ch, sendfd = -1, sendstatus; + pid_t sendpid = -1; + struct got_send_progress_arg spa; + int verbosity = 0, overwrite_refs = 0; + int send_all_branches = 0, send_all_tags = 0; + struct got_reference *ref = NULL; + + TAILQ_INIT(&branches); + TAILQ_INIT(&tags); + TAILQ_INIT(&all_branches); + TAILQ_INIT(&all_tags); + TAILQ_INIT(&delete_args); + TAILQ_INIT(&delete_branches); + + while ((ch = getopt(argc, argv, "ab:d:fr:t:Tvq")) != -1) { + switch (ch) { + case 'a': + send_all_branches = 1; + break; + case 'b': + error = got_pathlist_append(&branches, optarg, NULL); + if (error) + return error; + nbranches++; + break; + case 'd': + error = got_pathlist_append(&delete_args, optarg, NULL); + if (error) + return error; + break; + case 'f': + overwrite_refs = 1; + break; + case 'r': + repo_path = realpath(optarg, NULL); + if (repo_path == NULL) + return got_error_from_errno2("realpath", + optarg); + got_path_strip_trailing_slashes(repo_path); + break; + case 't': + error = got_pathlist_append(&tags, optarg, NULL); + if (error) + return error; + ntags++; + break; + case 'T': + send_all_tags = 1; + break; + case 'v': + if (verbosity < 0) + verbosity = 0; + else if (verbosity < 3) + verbosity++; + break; + case 'q': + verbosity = -1; + break; + default: + usage_send(); + /* NOTREACHED */ + } + } + argc -= optind; + argv += optind; + + if (send_all_branches && !TAILQ_EMPTY(&branches)) + option_conflict('a', 'b'); + if (send_all_tags && !TAILQ_EMPTY(&tags)) + option_conflict('T', 't'); + + + if (argc == 0) + remote_name = GOT_SEND_DEFAULT_REMOTE_NAME; + else if (argc == 1) + remote_name = argv[0]; + else + usage_send(); + + cwd = getcwd(NULL, 0); + if (cwd == NULL) { + error = got_error_from_errno("getcwd"); + goto done; + } + + if (repo_path == NULL) { + error = got_worktree_open(&worktree, cwd); + if (error && error->code != GOT_ERR_NOT_WORKTREE) + goto done; + else + error = NULL; + if (worktree) { + repo_path = + strdup(got_worktree_get_repo_path(worktree)); + if (repo_path == NULL) + error = got_error_from_errno("strdup"); + if (error) + goto done; + } else { + repo_path = strdup(cwd); + if (repo_path == NULL) { + error = got_error_from_errno("strdup"); + goto done; + } + } + } + + error = got_repo_open(&repo, repo_path, NULL); + if (error) + goto done; + + if (worktree) { + worktree_conf = got_worktree_get_gotconfig(worktree); + if (worktree_conf) { + got_gotconfig_get_remotes(&nremotes, &remotes, + worktree_conf); + for (i = 0; i < nremotes; i++) { + if (strcmp(remotes[i].name, remote_name) == 0) { + remote = &remotes[i]; + break; + } + } + } + } + if (remote == NULL) { + repo_conf = got_repo_get_gotconfig(repo); + if (repo_conf) { + got_gotconfig_get_remotes(&nremotes, &remotes, + repo_conf); + for (i = 0; i < nremotes; i++) { + if (strcmp(remotes[i].name, remote_name) == 0) { + remote = &remotes[i]; + break; + } + } + } + } + if (remote == NULL) { + got_repo_get_gitconfig_remotes(&nremotes, &remotes, repo); + for (i = 0; i < nremotes; i++) { + if (strcmp(remotes[i].name, remote_name) == 0) { + remote = &remotes[i]; + break; + } + } + } + if (remote == NULL) { + error = got_error_path(remote_name, GOT_ERR_NO_REMOTE); + goto done; + } + + error = got_fetch_parse_uri(&proto, &host, &port, &server_path, + &repo_name, remote->url); + if (error) + goto done; + + if (strcmp(proto, "git") == 0) { +#ifndef PROFILE + if (pledge("stdio rpath wpath cpath fattr flock proc exec " + "sendfd dns inet unveil", NULL) == -1) + err(1, "pledge"); +#endif + } else if (strcmp(proto, "git+ssh") == 0 || + strcmp(proto, "ssh") == 0) { +#ifndef PROFILE + if (pledge("stdio rpath wpath cpath fattr flock proc exec " + "sendfd unveil", NULL) == -1) + err(1, "pledge"); +#endif + } else if (strcmp(proto, "http") == 0 || + strcmp(proto, "git+http") == 0) { + error = got_error_path(proto, GOT_ERR_NOT_IMPL); + goto done; + } else { + error = got_error_path(proto, GOT_ERR_BAD_PROTO); + goto done; + } + + if (strcmp(proto, "git+ssh") == 0 || strcmp(proto, "ssh") == 0) { + if (unveil(GOT_FETCH_PATH_SSH, "x") != 0) { + error = got_error_from_errno2("unveil", + GOT_FETCH_PATH_SSH); + goto done; + } + } + error = apply_unveil(got_repo_get_path(repo), 0, NULL); + if (error) + goto done; + + if (send_all_branches) { + error = got_ref_list(&all_branches, repo, "refs/heads", + got_ref_cmp_by_name, NULL); + if (error) + goto done; + TAILQ_FOREACH(re, &all_branches, entry) { + const char *branchname = got_ref_get_name(re->ref); + error = got_pathlist_append(&branches, + branchname, NULL); + if (error) + goto done; + nbranches++; + } + } + + if (send_all_tags) { + error = got_ref_list(&all_tags, repo, "refs/tags", + got_ref_cmp_by_name, NULL); + if (error) + goto done; + TAILQ_FOREACH(re, &all_tags, entry) { + const char *tagname = got_ref_get_name(re->ref); + error = got_pathlist_append(&tags, + tagname, NULL); + if (error) + goto done; + ntags++; + } + } + + /* + * To prevent accidents only branches in refs/heads/ can be deleted + * with 'got send -d'. + * Deleting anything else requires local repository access or Git. + */ + TAILQ_FOREACH(pe, &delete_args, entry) { + const char *branchname = pe->path; + char *s; + struct got_pathlist_entry *new; + if (strncmp(branchname, "refs/heads/", 11) == 0) { + s = strdup(branchname); + if (s == NULL) { + error = got_error_from_errno("strdup"); + goto done; + } + } else { + if (asprintf(&s, "refs/heads/%s", branchname) == -1) { + error = got_error_from_errno("asprintf"); + goto done; + } + } + error = got_pathlist_insert(&new, &delete_branches, s, NULL); + if (error || new == NULL /* duplicate */) + free(s); + if (error) + goto done; + ndelete_branches++; + } + + if (nbranches == 0 && ndelete_branches == 0) { + struct got_reference *head_ref; + if (worktree) + error = got_ref_open(&head_ref, repo, + got_worktree_get_head_ref_name(worktree), 0); + else + error = got_ref_open(&head_ref, repo, GOT_REF_HEAD, 0); + if (error) + goto done; + if (got_ref_is_symbolic(head_ref)) { + error = got_ref_resolve_symbolic(&ref, repo, head_ref); + got_ref_close(head_ref); + if (error) + goto done; + } else + ref = head_ref; + error = got_pathlist_append(&branches, got_ref_get_name(ref), + NULL); + if (error) + goto done; + nbranches++; + } + + if (verbosity >= 0) + printf("Connecting to \"%s\" %s%s%s\n", remote->name, host, + port ? ":" : "", port ? port : ""); + + error = got_send_connect(&sendpid, &sendfd, proto, host, port, + server_path, verbosity); + if (error) + goto done; + + memset(&spa, 0, sizeof(spa)); + spa.last_scaled_packsize[0] = '\0'; + spa.last_p_deltify = -1; + spa.last_p_written = -1; + spa.verbosity = verbosity; + spa.delete_branches = &delete_branches; + error = got_send_pack(remote_name, &branches, &tags, &delete_branches, + verbosity, overwrite_refs, sendfd, repo, send_progress, &spa, + check_cancelled, NULL); + if (spa.printed_something) + putchar('\n'); + if (error) + goto done; + if (!spa.sent_something && verbosity >= 0) + printf("Already up-to-date\n"); +done: + if (sendpid > 0) { + if (kill(sendpid, SIGTERM) == -1) + error = got_error_from_errno("kill"); + if (waitpid(sendpid, &sendstatus, 0) == -1 && error == NULL) + error = got_error_from_errno("waitpid"); + } + if (sendfd != -1 && close(sendfd) == -1 && error == NULL) + error = got_error_from_errno("close"); + if (repo) { + const struct got_error *close_err = got_repo_close(repo); + if (error == NULL) + error = close_err; + } + if (worktree) + got_worktree_close(worktree); + if (ref) + got_ref_close(ref); + got_pathlist_free(&branches); + got_pathlist_free(&tags); + got_ref_list_free(&all_branches); + got_ref_list_free(&all_tags); + got_pathlist_free(&delete_args); + TAILQ_FOREACH(pe, &delete_branches, entry) + free((char *)pe->path); + got_pathlist_free(&delete_branches); + free(cwd); + free(repo_path); + free(proto); + free(host); + free(port); + free(server_path); + free(repo_name); + return error; +} + +__dead static void usage_cherrypick(void) { fprintf(stderr, "usage: %s cherrypick commit-id\n", getprogname()); blob - eab64a846eec63193c520e1b192077f1c38f24c9 blob + 5eb5cc5ebde432af823128fc7227bf48b866369a --- include/got_error.h +++ include/got_error.h @@ -148,6 +148,13 @@ #define GOT_ERR_CANNOT_PACK 131 #define GOT_ERR_LONELY_PACKIDX 132 #define GOT_ERR_OBJ_CSUM 133 +#define GOT_ERR_SEND_BAD_REF 134 +#define GOT_ERR_SEND_FAILED 135 +#define GOT_ERR_SEND_EMPTY 136 +#define GOT_ERR_SEND_ANCESTRY 137 +#define GOT_ERR_CAPA_DELETE_REFS 138 +#define GOT_ERR_SEND_DELETE_REF 139 +#define GOT_ERR_SEND_TAG_EXISTS 140 static const struct got_error { int code; @@ -304,6 +311,13 @@ static const struct got_error { { GOT_ERR_LONELY_PACKIDX, "pack index has no corresponding pack file; " "pack file must be restored or 'gotadmin cleanup -p' must be run" }, { GOT_ERR_OBJ_CSUM, "bad object checksum" }, + { GOT_ERR_SEND_BAD_REF, "reference cannot be sent" }, + { GOT_ERR_SEND_FAILED, "could not send pack file" }, + { GOT_ERR_SEND_EMPTY, "no references to send" }, + { GOT_ERR_SEND_ANCESTRY, "fetch and rebase required" }, + { GOT_ERR_CAPA_DELETE_REFS, "server cannot delete references" }, + { GOT_ERR_SEND_DELETE_REF, "reference cannot be deleted" }, + { GOT_ERR_SEND_TAG_EXISTS, "tag already exists on server" }, }; /* blob - /dev/null blob + 2f0388ea400f048c63027a0b901c892f672248bc (mode 644) --- /dev/null +++ include/got_send.h @@ -0,0 +1,72 @@ +/* + * Copyright (c) 2018, 2019 Ori Bernstein <ori@openbsd.org> + * Copyright (c) 2021 Stefan Sperling <stsp@openbsd.org> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +/* IANA assigned */ +#define GOT_DEFAULT_GIT_PORT 9418 +#define GOT_DEFAULT_GIT_PORT_STR "9418" + +#ifndef GOT_SEND_PATH_SSH +#define GOT_SEND_PATH_SSH "/usr/bin/ssh" +#endif + +#define GOT_SEND_DEFAULT_REMOTE_NAME "origin" + +#define GOT_SEND_PKTMAX 65536 + +/* + * Attempt to open a connection to a server using the provided protocol + * scheme, hostname port number (as a string) and server-side path. + * A verbosity level can be specified; it currently controls the amount + * of -v options passed to ssh(1). If the level is -1 ssh(1) will be run + * with the -q option. + * + * If successful return an open file descriptor for the connection which can + * be passed to other functions below, and must be disposed of with close(2). + * + * If an ssh(1) process was started return its PID as well, in which case + * the caller should eventually send SIGTERM to the procress and wait for + * the process to exit with waitpid(2). Otherwise, return PID -1. + */ +const struct got_error *got_send_connect(pid_t *, int *, const char *, + const char *, const char *, const char *, int); + +/* A callback function which gets invoked with progress information to print. */ +typedef const struct got_error *(*got_send_progress_cb)(void *, + off_t packfile_size, int ncommits, int nobj_total, + int nobj_deltify, int nobj_written, off_t bytes_sent, + const char *refname, int success); + +/* + * Attempt to generate a pack file and sent it to a server. + * This pack file will contain objects which are reachable in the local + * repository via the specified branches and tags. Any objects which are + * already present in the remote repository will be omitted from the + * pack file. + * + * If the server supports deletion of references, attempt to delete + * branches on the specified delete_branches list from the server. + * Such branches are not required to exist in the local repository. + * Requesting deletion of branches results in an error if the server + * does not support this feature. + */ +const struct got_error *got_send_pack(const char *remote_name, + struct got_pathlist_head *branch_names, + struct got_pathlist_head *tag_names, + struct got_pathlist_head *delete_branches, int verbosity, + int overwrite_refs, int sendfd, struct got_repository *repo, + got_send_progress_cb progress_cb, void *progress_arg, + got_cancel_cb cancel_cb, void *cancel_arg); blob - 180adaeec95923b909b3ce9e754eb670b166792d blob + 8a84ff3488e6ae227869888581b991d1ca4203c2 --- lib/got_lib_pack_create.h +++ lib/got_lib_pack_create.h @@ -24,6 +24,6 @@ const struct got_error *got_pack_create(uint8_t *pack_sha1, FILE *packfile, struct got_object_id **theirs, int ntheirs, struct got_object_id **ours, int nours, - struct got_repository *repo, int loose_obj_only, + struct got_repository *repo, int loose_obj_only, int allow_empty, got_pack_progress_cb progress_cb, void *progress_arg, got_cancel_cb cancel_cb, void *cancel_arg); blob - 6268cadc46c849d8772530b9e2fa38833090ea59 blob + daaf7f3639f1a1257da430a23bbc5529db74d685 --- lib/got_lib_privsep.h +++ lib/got_lib_privsep.h @@ -126,6 +126,14 @@ enum got_imsg_type { GOT_IMSG_IDXPACK_OUTFD, GOT_IMSG_IDXPACK_PROGRESS, GOT_IMSG_IDXPACK_DONE, + GOT_IMSG_SEND_REQUEST, + GOT_IMSG_SEND_REF, + GOT_IMSG_SEND_REMOTE_REF, + GOT_IMSG_SEND_REF_STATUS, + GOT_IMSG_SEND_PACK_REQUEST, + GOT_IMSG_SEND_PACKFD, + GOT_IMSG_SEND_UPLOAD_PROGRESS, + GOT_IMSG_SEND_DONE, /* Messages related to pack files. */ GOT_IMSG_PACKIDX, @@ -335,6 +343,41 @@ struct got_imsg_fetch_download_progress { off_t packfile_bytes; }; +/* Structure for GOT_IMSG_SEND_REQUEST data. */ +struct got_imsg_send_request { + int verbosity; + size_t nrefs; + /* Followed by nrefs GOT_IMSG_SEND_REF messages. */ +} __attribute__((__packed__)); + +/* Structure for GOT_IMSG_SEND_UPLOAD_PROGRESS data. */ +struct got_imsg_send_upload_progress { + /* Number of packfile data bytes uploaded so far. */ + off_t packfile_bytes; +}; + +/* Structure for GOT_IMSG_SEND_REF data. */ +struct got_imsg_send_ref { + uint8_t id[SHA1_DIGEST_LENGTH]; + int delete; + size_t name_len; + /* Followed by name_len data bytes. */ +} __attribute__((__packed__)); + +/* Structure for GOT_IMSG_SEND_REMOTE_REF data. */ +struct got_imsg_send_remote_ref { + uint8_t id[SHA1_DIGEST_LENGTH]; + size_t name_len; + /* Followed by name_len data bytes. */ +} __attribute__((__packed__)); + +/* Structure for GOT_IMSG_SEND_REF_STATUS data. */ +struct got_imsg_send_ref_status { + int success; + size_t name_len; + /* Followed by name_len data bytes. */ +} __attribute__((__packed__)); + /* Structure for GOT_IMSG_IDXPACK_REQUEST data. */ struct got_imsg_index_pack_request { uint8_t pack_hash[SHA1_DIGEST_LENGTH]; @@ -466,6 +509,13 @@ const struct got_error *got_privsep_send_fetch_outfd(s const struct got_error *got_privsep_recv_fetch_progress(int *, struct got_object_id **, char **, struct got_pathlist_head *, char **, off_t *, uint8_t *, struct imsgbuf *); +const struct got_error *got_privsep_send_send_req(struct imsgbuf *, int, + struct got_pathlist_head *, struct got_pathlist_head *, int); +const struct got_error *got_privsep_recv_send_remote_refs( + struct got_pathlist_head *, struct imsgbuf *); +const struct got_error *got_privsep_send_packfd(struct imsgbuf *, int); +const struct got_error *got_privsep_recv_send_progress(int *, off_t *, + int *, char **, struct imsgbuf *); const struct got_error *got_privsep_get_imsg_obj(struct got_object **, struct imsg *, struct imsgbuf *); const struct got_error *got_privsep_recv_obj(struct got_object **, blob - 73b754dfefc5345d57bb8ffe4d81d0b0f8859745 blob + 1455b6df378eb19d397ce2e3d3588af5a0256490 --- lib/pack_create.c +++ lib/pack_create.c @@ -1271,7 +1271,7 @@ const struct got_error * got_pack_create(uint8_t *packsha1, FILE *packfile, struct got_object_id **theirs, int ntheirs, struct got_object_id **ours, int nours, - struct got_repository *repo, int loose_obj_only, + struct got_repository *repo, int loose_obj_only, int allow_empty, got_pack_progress_cb progress_cb, void *progress_arg, got_cancel_cb cancel_cb, void *cancel_arg) { @@ -1284,16 +1284,17 @@ got_pack_create(uint8_t *packsha1, FILE *packfile, if (err) return err; - if (nmeta == 0) { + if (nmeta == 0 && !allow_empty) { err = got_error(GOT_ERR_CANNOT_PACK); goto done; } + if (nmeta > 0) { + err = pick_deltas(meta, nmeta, nours, repo, + progress_cb, progress_arg, cancel_cb, cancel_arg); + if (err) + goto done; + } - err = pick_deltas(meta, nmeta, nours, repo, - progress_cb, progress_arg, cancel_cb, cancel_arg); - if (err) - goto done; - err = genpack(packsha1, packfile, meta, nmeta, nours, 1, repo, progress_cb, progress_arg, cancel_cb, cancel_arg); if (err) blob - 274f3c3fcbe691037dcbe34aba718d4c708a4af5 blob + ecad96e05628c923f414e7613a59a1e54b82bba0 --- lib/privsep.c +++ lib/privsep.c @@ -862,7 +862,250 @@ done: return err; } +static const struct got_error * +send_send_ref(const char *name, size_t name_len, struct got_object_id *id, + int delete, struct imsgbuf *ibuf) +{ + const struct got_error *err = NULL; + size_t len; + struct ibuf *wbuf; + + len = sizeof(struct got_imsg_send_ref) + name_len; + wbuf = imsg_create(ibuf, GOT_IMSG_SEND_REF, 0, 0, len); + if (wbuf == NULL) + return got_error_from_errno("imsg_create SEND_REF"); + + /* Keep in sync with struct got_imsg_send_ref! */ + if (imsg_add(wbuf, id->sha1, sizeof(id->sha1)) == -1) { + err = got_error_from_errno("imsg_add SEND_REF"); + ibuf_free(wbuf); + return err; + } + if (imsg_add(wbuf, &delete, sizeof(delete)) == -1) { + err = got_error_from_errno("imsg_add SEND_REF"); + ibuf_free(wbuf); + return err; + } + if (imsg_add(wbuf, &name_len, sizeof(name_len)) == -1) { + err = got_error_from_errno("imsg_add SEND_REF"); + ibuf_free(wbuf); + return err; + } + if (imsg_add(wbuf, name, name_len) == -1) { + err = got_error_from_errno("imsg_add SEND_REF"); + ibuf_free(wbuf); + return err; + } + + wbuf->fd = -1; + imsg_close(ibuf, wbuf); + return flush_imsg(ibuf); +} + const struct got_error * +got_privsep_send_send_req(struct imsgbuf *ibuf, int fd, + struct got_pathlist_head *have_refs, + struct got_pathlist_head *delete_refs, + int verbosity) +{ + const struct got_error *err = NULL; + struct got_pathlist_entry *pe; + struct got_imsg_send_request sendreq; + struct got_object_id zero_id; + + memset(&zero_id, 0, sizeof(zero_id)); + memset(&sendreq, 0, sizeof(sendreq)); + sendreq.verbosity = verbosity; + TAILQ_FOREACH(pe, have_refs, entry) + sendreq.nrefs++; + TAILQ_FOREACH(pe, delete_refs, entry) + sendreq.nrefs++; + if (imsg_compose(ibuf, GOT_IMSG_SEND_REQUEST, 0, 0, fd, + &sendreq, sizeof(sendreq)) == -1) { + err = got_error_from_errno( + "imsg_compose FETCH_SERVER_PROGRESS"); + goto done; + } + + err = flush_imsg(ibuf); + if (err) + goto done; + fd = -1; + + TAILQ_FOREACH(pe, have_refs, entry) { + const char *name = pe->path; + size_t name_len = pe->path_len; + struct got_object_id *id = pe->data; + err = send_send_ref(name, name_len, id, 0, ibuf); + if (err) + goto done; + } + + TAILQ_FOREACH(pe, delete_refs, entry) { + const char *name = pe->path; + size_t name_len = pe->path_len; + err = send_send_ref(name, name_len, &zero_id, 1, ibuf); + if (err) + goto done; + } +done: + if (fd != -1 && close(fd) == -1 && err == NULL) + err = got_error_from_errno("close"); + return err; + +} + +const struct got_error * +got_privsep_recv_send_remote_refs(struct got_pathlist_head *remote_refs, + struct imsgbuf *ibuf) +{ + const struct got_error *err = NULL; + struct imsg imsg; + size_t datalen; + int done = 0; + struct got_imsg_send_remote_ref iremote_ref; + struct got_object_id *id = NULL; + char *refname = NULL; + struct got_pathlist_entry *new; + + while (!done) { + err = got_privsep_recv_imsg(&imsg, ibuf, 0); + if (err) + return err; + datalen = imsg.hdr.len - IMSG_HEADER_SIZE; + switch (imsg.hdr.type) { + case GOT_IMSG_ERROR: + if (datalen < sizeof(struct got_imsg_error)) { + err = got_error(GOT_ERR_PRIVSEP_LEN); + goto done; + } + err = recv_imsg_error(&imsg, datalen); + goto done; + case GOT_IMSG_SEND_REMOTE_REF: + if (datalen < sizeof(iremote_ref)) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + goto done; + } + memcpy(&iremote_ref, imsg.data, sizeof(iremote_ref)); + if (datalen != sizeof(iremote_ref) + + iremote_ref.name_len) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + goto done; + } + id = malloc(sizeof(*id)); + if (id == NULL) { + err = got_error_from_errno("malloc"); + goto done; + } + memcpy(id->sha1, iremote_ref.id, SHA1_DIGEST_LENGTH); + refname = strndup(imsg.data + sizeof(iremote_ref), + datalen - sizeof(iremote_ref)); + if (refname == NULL) { + err = got_error_from_errno("strndup"); + goto done; + } + err = got_pathlist_insert(&new, remote_refs, + refname, id); + if (err) + goto done; + if (new == NULL) { /* duplicate which wasn't inserted */ + free(id); + free(refname); + } + id = NULL; + refname = NULL; + break; + case GOT_IMSG_SEND_PACK_REQUEST: + if (datalen != 0) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + goto done; + } + /* got-send-pack is now waiting for a pack file. */ + done = 1; + break; + default: + err = got_error(GOT_ERR_PRIVSEP_MSG); + break; + } + } +done: + free(id); + free(refname); + imsg_free(&imsg); + return err; +} + +const struct got_error * +got_privsep_send_packfd(struct imsgbuf *ibuf, int fd) +{ + return send_fd(ibuf, GOT_IMSG_SEND_PACKFD, fd); +} + +const struct got_error * +got_privsep_recv_send_progress(int *done, off_t *bytes_sent, + int *success, char **refname, struct imsgbuf *ibuf) +{ + const struct got_error *err = NULL; + struct imsg imsg; + size_t datalen; + struct got_imsg_send_ref_status iref_status; + + /* Do not reset the current value of 'bytes_sent', it accumulates. */ + *done = 0; + *success = 0; + *refname = NULL; + + err = got_privsep_recv_imsg(&imsg, ibuf, 0); + if (err) + return err; + + datalen = imsg.hdr.len - IMSG_HEADER_SIZE; + switch (imsg.hdr.type) { + case GOT_IMSG_ERROR: + if (datalen < sizeof(struct got_imsg_error)) { + err = got_error(GOT_ERR_PRIVSEP_LEN); + break; + } + err = recv_imsg_error(&imsg, datalen); + break; + case GOT_IMSG_SEND_UPLOAD_PROGRESS: + if (datalen < sizeof(*bytes_sent)) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + break; + } + memcpy(bytes_sent, imsg.data, sizeof(*bytes_sent)); + break; + case GOT_IMSG_SEND_REF_STATUS: + if (datalen < sizeof(iref_status)) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + break; + } + memcpy(&iref_status, imsg.data, sizeof(iref_status)); + if (datalen != sizeof(iref_status) + iref_status.name_len) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + break; + } + *success = iref_status.success; + *refname = strndup(imsg.data + sizeof(iref_status), + iref_status.name_len); + break; + case GOT_IMSG_SEND_DONE: + if (datalen != 0) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + break; + } + *done = 1; + break; + default: + err = got_error(GOT_ERR_PRIVSEP_MSG); + break; + } + + imsg_free(&imsg); + return err; +} + +const struct got_error * got_privsep_send_index_pack_req(struct imsgbuf *ibuf, uint8_t *pack_sha1, int fd) { @@ -2451,6 +2694,7 @@ got_privsep_unveil_exec_helpers(void) GOT_PATH_PROG_READ_GOTCONFIG, GOT_PATH_PROG_FETCH_PACK, GOT_PATH_PROG_INDEX_PACK, + GOT_PATH_PROG_SEND_PACK, }; size_t i; blob - deb76e2c5553e4ba47753fc3862764b88217ec6a blob + cf1fd8aad18ee23db95c5c4e6417a650c9b5b15d --- lib/repository_admin.c +++ lib/repository_admin.c @@ -198,7 +198,7 @@ got_repo_pack_objects(FILE **packfile, struct got_obje } err = got_pack_create((*pack_hash)->sha1, *packfile, theirs, ntheirs, - ours, nours, repo, loose_obj_only, progress_cb, progress_arg, + ours, nours, repo, loose_obj_only, 0, progress_cb, progress_arg, cancel_cb, cancel_arg); if (err) goto done; blob - /dev/null blob + 52f062e8ad4e3182b002248a00f2c089435712f0 (mode 644) --- /dev/null +++ lib/send.c @@ -0,0 +1,841 @@ +/* + * Copyright (c) 2018, 2019 Ori Bernstein <ori@openbsd.org> + * Copyright (c) 2021 Stefan Sperling <stsp@openbsd.org> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include <sys/types.h> +#include <sys/stat.h> +#include <sys/queue.h> +#include <sys/uio.h> +#include <sys/socket.h> +#include <sys/wait.h> +#include <sys/resource.h> +#include <sys/socket.h> + +#include <endian.h> +#include <errno.h> +#include <err.h> +#include <fcntl.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <stdint.h> +#include <sha1.h> +#include <unistd.h> +#include <zlib.h> +#include <ctype.h> +#include <limits.h> +#include <imsg.h> +#include <time.h> +#include <uuid.h> +#include <netdb.h> +#include <netinet/in.h> + +#include "got_error.h" +#include "got_reference.h" +#include "got_repository.h" +#include "got_path.h" +#include "got_cancel.h" +#include "got_worktree.h" +#include "got_object.h" +#include "got_opentemp.h" +#include "got_send.h" +#include "got_repository_admin.h" +#include "got_commit_graph.h" + +#include "got_lib_delta.h" +#include "got_lib_inflate.h" +#include "got_lib_object.h" +#include "got_lib_object_parse.h" +#include "got_lib_object_create.h" +#include "got_lib_pack.h" +#include "got_lib_sha1.h" +#include "got_lib_privsep.h" +#include "got_lib_object_cache.h" +#include "got_lib_repository.h" +#include "got_lib_pack_create.h" + +#ifndef nitems +#define nitems(_a) (sizeof((_a)) / sizeof((_a)[0])) +#endif + +#ifndef ssizeof +#define ssizeof(_x) ((ssize_t)(sizeof(_x))) +#endif + +#ifndef MIN +#define MIN(_a,_b) ((_a) < (_b) ? (_a) : (_b)) +#endif + +static const struct got_error * +dial_ssh(pid_t *sendpid, int *sendfd, const char *host, const char *port, + const char *path, const char *direction, int verbosity) +{ + const struct got_error *error = NULL; + int pid, pfd[2]; + char cmd[64]; + char *argv[11]; + int i = 0, j; + + *sendpid = -1; + *sendfd = -1; + + argv[i++] = GOT_SEND_PATH_SSH; + if (port != NULL) { + argv[i++] = "-p"; + argv[i++] = (char *)port; + } + if (verbosity == -1) { + argv[i++] = "-q"; + } else { + /* ssh(1) allows up to 3 "-v" options. */ + for (j = 0; j < MIN(3, verbosity); j++) + argv[i++] = "-v"; + } + argv[i++] = "--"; + argv[i++] = (char *)host; + argv[i++] = (char *)cmd; + argv[i++] = (char *)path; + argv[i++] = NULL; + + if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, pfd) == -1) + return got_error_from_errno("socketpair"); + + pid = fork(); + if (pid == -1) { + error = got_error_from_errno("fork"); + close(pfd[0]); + close(pfd[1]); + return error; + } else if (pid == 0) { + int n; + if (close(pfd[1]) == -1) + err(1, "close"); + if (dup2(pfd[0], 0) == -1) + err(1, "dup2"); + if (dup2(pfd[0], 1) == -1) + err(1, "dup2"); + n = snprintf(cmd, sizeof(cmd), "git-%s-pack", direction); + if (n < 0 || n >= ssizeof(cmd)) + err(1, "snprintf"); + if (execv(GOT_SEND_PATH_SSH, argv) == -1) + err(1, "execv"); + abort(); /* not reached */ + } else { + if (close(pfd[0]) == -1) + return got_error_from_errno("close"); + *sendpid = pid; + *sendfd = pfd[1]; + return NULL; + } +} + +static const struct got_error * +dial_git(int *sendfd, const char *host, const char *port, const char *path, + const char *direction) +{ + const struct got_error *err = NULL; + struct addrinfo hints, *servinfo, *p; + char *cmd = NULL; + int fd = -1, len, r, eaicode; + + *sendfd = -1; + + if (port == NULL) + port = GOT_DEFAULT_GIT_PORT_STR; + + memset(&hints, 0, sizeof hints); + hints.ai_family = AF_UNSPEC; + hints.ai_socktype = SOCK_STREAM; + eaicode = getaddrinfo(host, port, &hints, &servinfo); + if (eaicode) { + char msg[512]; + snprintf(msg, sizeof(msg), "%s: %s", host, + gai_strerror(eaicode)); + return got_error_msg(GOT_ERR_ADDRINFO, msg); + } + + for (p = servinfo; p != NULL; p = p->ai_next) { + if ((fd = socket(p->ai_family, p->ai_socktype, + p->ai_protocol)) == -1) + continue; + if (connect(fd, p->ai_addr, p->ai_addrlen) == 0) { + err = NULL; + break; + } + err = got_error_from_errno("connect"); + close(fd); + } + if (p == NULL) + goto done; + + if (asprintf(&cmd, "git-%s-pack %s", direction, path) == -1) { + err = got_error_from_errno("asprintf"); + goto done; + } + len = 4 + strlen(cmd) + 1 + strlen("host=") + strlen(host) + 1; + r = dprintf(fd, "%04x%s%chost=%s%c", len, cmd, '\0', host, '\0'); + if (r < 0) + err = got_error_from_errno("dprintf"); +done: + free(cmd); + if (err) { + if (fd != -1) + close(fd); + } else + *sendfd = fd; + return err; +} + +const struct got_error * +got_send_connect(pid_t *sendpid, int *sendfd, const char *proto, + const char *host, const char *port, const char *server_path, int verbosity) +{ + const struct got_error *err = NULL; + + *sendpid = -1; + *sendfd = -1; + + if (strcmp(proto, "ssh") == 0 || strcmp(proto, "git+ssh") == 0) + err = dial_ssh(sendpid, sendfd, host, port, server_path, + "receive", verbosity); + else if (strcmp(proto, "git") == 0) + err = dial_git(sendfd, host, port, server_path, "receive"); + else if (strcmp(proto, "http") == 0 || strcmp(proto, "git+http") == 0) + err = got_error_path(proto, GOT_ERR_NOT_IMPL); + else + err = got_error_path(proto, GOT_ERR_BAD_PROTO); + return err; +} + +struct pack_progress_arg { + got_send_progress_cb progress_cb; + void *progress_arg; + + off_t packfile_size; + int ncommits; + int nobj_total; + int nobj_deltify; + int nobj_written; +}; + +static const struct got_error * +pack_progress(void *arg, off_t packfile_size, int ncommits, + int nobj_total, int nobj_deltify, int nobj_written) +{ + const struct got_error *err; + struct pack_progress_arg *a = arg; + + err = a->progress_cb(a->progress_arg, packfile_size, ncommits, + nobj_total, nobj_deltify, nobj_written, 0, NULL, 0); + if (err) + return err; + + a->packfile_size = packfile_size; + a->ncommits = ncommits; + a->nobj_total = nobj_total; + a->nobj_deltify = nobj_deltify; + a->nobj_written = nobj_written; + return NULL; +} + +static const struct got_error * +insert_ref(struct got_reflist_head *refs, const char *refname, + struct got_repository *repo) +{ + const struct got_error *err; + struct got_reference *ref; + struct got_reflist_entry *new; + + err = got_ref_open(&ref, repo, refname, 0); + if (err) + return err; + + err = got_reflist_insert(&new, refs, ref, got_ref_cmp_by_name, NULL); + if (err || new == NULL /* duplicate */) + got_ref_close(ref); + + return err; +} + +static const struct got_error * +check_linear_ancestry(const char *refname, struct got_object_id *my_id, + struct got_object_id *their_id, struct got_repository *repo, + got_cancel_cb cancel_cb, void *cancel_arg) +{ + const struct got_error *err = NULL; + struct got_object_id *yca_id; + int obj_type; + + err = got_object_get_type(&obj_type, repo, their_id); + if (err) + return err; + if (obj_type != GOT_OBJ_TYPE_COMMIT) + return got_error_fmt(GOT_ERR_OBJ_TYPE, + "bad object type on server for %s", refname); + + err = got_commit_graph_find_youngest_common_ancestor(&yca_id, + my_id, their_id, repo, cancel_cb, cancel_arg); + if (err) + return err; + if (yca_id == NULL) + return got_error_fmt(GOT_ERR_SEND_ANCESTRY, "%s", refname); + + /* + * Require a straight line of history between the two commits, + * with their commit being older than my commit. + * + * Non-linear situations such as this require a rebase: + * + * (theirs) D F (mine) + * \ / + * C E + * \ / + * B (yca) + * | + * A + */ + if (got_object_id_cmp(their_id, yca_id) != 0) + err = got_error_fmt(GOT_ERR_SEND_ANCESTRY, "%s", refname); + + free(yca_id); + return err; +} + +static const struct got_error * +realloc_ids(struct got_object_id ***ids, size_t *nalloc, size_t n) +{ + struct got_object_id **new; + const size_t alloc_chunksz = 256; + + if (*nalloc >= n + alloc_chunksz) + return NULL; + + new = recallocarray(*ids, *nalloc, *nalloc + alloc_chunksz, + sizeof(struct got_object_id)); + if (new == NULL) + return got_error_from_errno("recallocarray"); + + *ids = new; + *nalloc += alloc_chunksz; + return NULL; +} + +static struct got_reference * +find_ref(struct got_reflist_head *refs, const char *refname) +{ + struct got_reflist_entry *re; + + TAILQ_FOREACH(re, refs, entry) { + if (got_path_cmp(got_ref_get_name(re->ref), refname, + strlen(got_ref_get_name(re->ref)), + strlen(refname)) == 0) { + return re->ref; + } + } + + return NULL; +} + +static struct got_pathlist_entry * +find_their_ref(struct got_pathlist_head *their_refs, const char *refname) +{ + struct got_pathlist_entry *pe; + + TAILQ_FOREACH(pe, their_refs, entry) { + const char *their_refname = pe->path; + if (got_path_cmp(their_refname, refname, + strlen(their_refname), strlen(refname)) == 0) { + return pe; + } + } + + return NULL; +} + +static const struct got_error * +get_remote_refname(char **remote_refname, const char *remote_name, + const char *refname) +{ + if (strncmp(refname, "refs/", 5) == 0) + refname += 5; + if (strncmp(refname, "heads/", 6) == 0) + refname += 6; + + if (asprintf(remote_refname, "refs/remotes/%s/%s", + remote_name, refname) == -1) + return got_error_from_errno("asprintf"); + + return NULL; +} + +static const struct got_error * +update_remote_ref(struct got_reference *my_ref, const char *remote_name, + struct got_repository *repo) +{ + const struct got_error *err, *unlock_err; + struct got_object_id *my_id; + struct got_reference *ref = NULL; + char *remote_refname = NULL; + int ref_locked = 0; + + err = got_ref_resolve(&my_id, repo, my_ref); + if (err) + return err; + + err = get_remote_refname(&remote_refname, remote_name, + got_ref_get_name(my_ref)); + if (err) + goto done; + + err = got_ref_open(&ref, repo, remote_refname, 1 /* lock */); + if (err) { + if (err->code != GOT_ERR_NOT_REF) + goto done; + err = got_ref_alloc(&ref, remote_refname, my_id); + if (err) + goto done; + } else { + ref_locked = 1; + err = got_ref_change_ref(ref, my_id); + if (err) + goto done; + } + + err = got_ref_write(ref, repo); +done: + if (ref) { + if (ref_locked) { + unlock_err = got_ref_unlock(ref); + if (unlock_err && err == NULL) + err = unlock_err; + } + got_ref_close(ref); + } + free(my_id); + free(remote_refname); + return err; +} + +const struct got_error* +got_send_pack(const char *remote_name, struct got_pathlist_head *branch_names, + struct got_pathlist_head *tag_names, + struct got_pathlist_head *delete_branches, + int verbosity, int overwrite_refs, int sendfd, + struct got_repository *repo, got_send_progress_cb progress_cb, + void *progress_arg, got_cancel_cb cancel_cb, void *cancel_arg) +{ + int imsg_sendfds[2]; + int npackfd = -1, nsendfd = -1; + int sendstatus, done = 0; + const struct got_error *err; + struct imsgbuf sendibuf; + pid_t sendpid = -1; + struct got_reflist_head refs; + struct got_pathlist_head have_refs; + struct got_pathlist_head their_refs; + struct got_pathlist_entry *pe; + struct got_reflist_entry *re; + struct got_object_id **our_ids = NULL; + struct got_object_id **their_ids = NULL; + struct got_object_id *my_id = NULL; + int i, nours = 0, ntheirs = 0; + size_t nalloc_ours = 0, nalloc_theirs = 0; + int refs_to_send = 0; + off_t bytes_sent = 0; + struct pack_progress_arg ppa; + uint8_t packsha1[SHA1_DIGEST_LENGTH]; + FILE *packfile = NULL; + + TAILQ_INIT(&refs); + TAILQ_INIT(&have_refs); + TAILQ_INIT(&their_refs); + + TAILQ_FOREACH(pe, branch_names, entry) { + const char *branchname = pe->path; + if (strncmp(branchname, "refs/heads/", 11) != 0) { + char *s; + if (asprintf(&s, "refs/heads/%s", branchname) == -1) { + err = got_error_from_errno("asprintf"); + goto done; + } + err = insert_ref(&refs, s, repo); + free(s); + } else { + err = insert_ref(&refs, branchname, repo); + } + if (err) + goto done; + } + + TAILQ_FOREACH(pe, delete_branches, entry) { + const char *branchname = pe->path; + struct got_reference *ref; + if (strncmp(branchname, "refs/heads/", 11) != 0) { + err = got_error_fmt(GOT_ERR_SEND_DELETE_REF, "%s", + branchname); + goto done; + } + ref = find_ref(&refs, branchname); + if (ref) { + err = got_error_fmt(GOT_ERR_SEND_DELETE_REF, + "changes on %s will be sent to server", + branchname); + goto done; + } + } + + TAILQ_FOREACH(pe, tag_names, entry) { + const char *tagname = pe->path; + if (strncmp(tagname, "refs/tags/", 10) != 0) { + char *s; + if (asprintf(&s, "refs/tags/%s", tagname) == -1) { + err = got_error_from_errno("asprintf"); + goto done; + } + err = insert_ref(&refs, s, repo); + free(s); + } else { + err = insert_ref(&refs, tagname, repo); + } + if (err) + goto done; + } + + if (TAILQ_EMPTY(&refs) && TAILQ_EMPTY(delete_branches)) { + err = got_error(GOT_ERR_SEND_EMPTY); + goto done; + } + + TAILQ_FOREACH(re, &refs, entry) { + struct got_object_id *id; + int obj_type; + + if (got_ref_is_symbolic(re->ref)) { + err = got_error_fmt(GOT_ERR_BAD_REF_TYPE, + "cannot send symbolic reference %s", + got_ref_get_name(re->ref)); + goto done; + } + + err = got_ref_resolve(&id, repo, re->ref); + if (err) + goto done; + err = got_object_get_type(&obj_type, repo, id); + free(id); + if (err) + goto done; + switch (obj_type) { + case GOT_OBJ_TYPE_COMMIT: + case GOT_OBJ_TYPE_TAG: + break; + default: + err = got_error_fmt(GOT_ERR_OBJ_TYPE, + "cannot send %s", got_ref_get_name(re->ref)); + goto done; + } + } + + packfile = got_opentemp(); + if (packfile == NULL) { + err = got_error_from_errno("got_opentemp"); + goto done; + } + + err = realloc_ids(&our_ids, &nalloc_ours, 0); + if (err) + goto done; + err = realloc_ids(&their_ids, &nalloc_ours, 0); + if (err) + goto done; + + if (socketpair(AF_UNIX, SOCK_STREAM, PF_UNSPEC, imsg_sendfds) == -1) { + err = got_error_from_errno("socketpair"); + goto done; + } + + sendpid = fork(); + if (sendpid == -1) { + err = got_error_from_errno("fork"); + goto done; + } else if (sendpid == 0){ + got_privsep_exec_child(imsg_sendfds, + GOT_PATH_PROG_SEND_PACK, got_repo_get_path(repo)); + } + + if (close(imsg_sendfds[1]) == -1) { + err = got_error_from_errno("close"); + goto done; + } + imsg_init(&sendibuf, imsg_sendfds[0]); + nsendfd = dup(sendfd); + if (nsendfd == -1) { + err = got_error_from_errno("dup"); + goto done; + } + + /* + * Convert reflist to pathlist since the privsep layer + * is linked into helper programs which lack reference.c. + */ + TAILQ_FOREACH(re, &refs, entry) { + struct got_object_id *id; + err = got_ref_resolve(&id, repo, re->ref); + if (err) + goto done; + err = got_pathlist_append(&have_refs, + got_ref_get_name(re->ref), id); + if (err) + goto done; + /* + * Also prepare the array of our object IDs which + * will be needed for generating a pack file. + */ + err = realloc_ids(&our_ids, &nalloc_ours, nours); + if (err) + goto done; + our_ids[nours] = id; + nours++; + } + + err = got_privsep_send_send_req(&sendibuf, nsendfd, &have_refs, + delete_branches, verbosity); + if (err) + goto done; + nsendfd = -1; + + err = got_privsep_recv_send_remote_refs(&their_refs, &sendibuf); + if (err) + goto done; + + /* + * Process references reported by the server. + * Push appropriate object IDs onto the "their IDs" array. + * This array will be used to exclude objects which already + * exist on the server from our pack file. + */ + TAILQ_FOREACH(pe, &their_refs, entry) { + const char *refname = pe->path; + struct got_object_id *their_id = pe->data; + int have_their_id; + struct got_object *obj; + struct got_reference *my_ref = NULL; + int is_tag = 0; + + /* Don't blindly trust the server to send us valid names. */ + if (!got_ref_name_is_valid(refname)) + continue; + + /* + * Find out whether this is a reference we want to upload. + * Otherwise we can still use this reference as a hint to + * avoid uploading any objects the server already has. + */ + my_ref = find_ref(&refs, refname); + if (my_ref) { + err = got_ref_resolve(&my_id, repo, my_ref); + if (err) + goto done; + if (got_object_id_cmp(my_id, their_id) == 0) { + free(my_id); + my_id = NULL; + continue; + } + refs_to_send++; + + } + + if (strncmp(refname, "refs/tags/", 10) == 0) + is_tag = 1; + + /* Prevent tags from being overwritten by default. */ + if (!overwrite_refs && my_ref && is_tag) { + err = got_error_fmt(GOT_ERR_SEND_TAG_EXISTS, + "%s", refname); + goto done; + } + + /* Check if their object exists locally. */ + err = got_object_open(&obj, repo, their_id); + if (err) { + if (err->code != GOT_ERR_NO_OBJ) + goto done; + if (!overwrite_refs && my_ref != NULL) { + err = got_error_fmt(GOT_ERR_SEND_ANCESTRY, + "%s", refname); + goto done; + } + have_their_id = 0; + } else { + got_object_close(obj); + have_their_id = 1; + } + + err = realloc_ids(&their_ids, &nalloc_theirs, ntheirs); + if (err) + goto done; + + if (have_their_id) { + /* Enforce linear ancestry if required. */ + if (!overwrite_refs && my_ref && !is_tag) { + struct got_object_id *my_id; + err = got_ref_resolve(&my_id, repo, my_ref); + if (err) + goto done; + err = check_linear_ancestry(refname, my_id, + their_id, repo, cancel_cb, cancel_arg); + free(my_id); + my_id = NULL; + if (err) + goto done; + } + /* Exclude any objects reachable via their ID. */ + their_ids[ntheirs] = got_object_id_dup(their_id); + if (their_ids[ntheirs] == NULL) { + err = got_error_from_errno("got_object_id_dup"); + goto done; + } + ntheirs++; + } else if (!is_tag) { + char *remote_refname; + struct got_reference *ref; + /* + * Exclude any objects which exist on the server + * according to a locally cached remote reference. + */ + err = get_remote_refname(&remote_refname, + remote_name, refname); + if (err) + goto done; + err = got_ref_open(&ref, repo, remote_refname, 0); + free(remote_refname); + if (err) { + if (err->code != GOT_ERR_NOT_REF) + goto done; + } else { + err = got_ref_resolve(&their_ids[ntheirs], + repo, ref); + got_ref_close(ref); + if (err) + goto done; + ntheirs++; + } + } + } + + /* Account for any new references we are going to upload. */ + TAILQ_FOREACH(re, &refs, entry) { + if (find_their_ref(&their_refs, + got_ref_get_name(re->ref)) == NULL) + refs_to_send++; + } + + if (refs_to_send == 0) { + got_privsep_send_stop(imsg_sendfds[0]); + goto done; + } + + memset(&ppa, 0, sizeof(ppa)); + ppa.progress_cb = progress_cb; + ppa.progress_arg = progress_arg; + err = got_pack_create(packsha1, packfile, their_ids, ntheirs, + our_ids, nours, repo, 0, 1, pack_progress, &ppa, + cancel_cb, cancel_arg); + if (err) + goto done; + + if (fflush(packfile) == -1) { + err = got_error_from_errno("fflush"); + goto done; + } + + npackfd = dup(fileno(packfile)); + if (npackfd == -1) { + err = got_error_from_errno("dup"); + goto done; + } + err = got_privsep_send_packfd(&sendibuf, npackfd); + if (err != NULL) + goto done; + npackfd = -1; + + while (!done) { + int success = 0; + char *refname = NULL; + off_t bytes_sent_cur = 0; + if (cancel_cb) { + err = (*cancel_cb)(cancel_arg); + if (err) + goto done; + } + err = got_privsep_recv_send_progress(&done, &bytes_sent, + &success, &refname, &sendibuf); + if (err) + goto done; + if (refname && got_ref_name_is_valid(refname) && success && + strncmp(refname, "refs/tags/", 10) != 0) { + struct got_reference *my_ref; + /* + * The server has accepted our changes. + * Update our reference in refs/remotes/ accordingly. + */ + my_ref = find_ref(&refs, refname); + if (my_ref) { + err = update_remote_ref(my_ref, remote_name, + repo); + if (err) + goto done; + } + } + if (refname != NULL || + bytes_sent_cur != bytes_sent) { + err = progress_cb(progress_arg, ppa.packfile_size, + ppa.ncommits, ppa.nobj_total, ppa.nobj_deltify, + ppa.nobj_written, bytes_sent, + refname, success); + if (err) { + free(refname); + goto done; + } + bytes_sent_cur = bytes_sent; + } + free(refname); + } +done: + if (sendpid != -1) { + if (err) + got_privsep_send_stop(imsg_sendfds[0]); + if (waitpid(sendpid, &sendstatus, 0) == -1 && err == NULL) + err = got_error_from_errno("waitpid"); + } + if (packfile && fclose(packfile) == EOF && err == NULL) + err = got_error_from_errno("fclose"); + if (nsendfd != -1 && close(nsendfd) == -1 && err == NULL) + err = got_error_from_errno("close"); + if (npackfd != -1 && close(npackfd) == -1 && err == NULL) + err = got_error_from_errno("close"); + + got_ref_list_free(&refs); + got_pathlist_free(&have_refs); + got_pathlist_free(&their_refs); + for (i = 0; i < nours; i++) + free(our_ids[i]); + free(our_ids); + for (i = 0; i < ntheirs; i++) + free(their_ids[i]); + free(their_ids); + free(my_id); + return err; +} blob - 1e55c9808beb7f0984445d4aae36eaddd0ceb5d4 blob + 3783b56689f6ab58fbacbd8f0f990a7154d90f61 --- libexec/Makefile +++ libexec/Makefile @@ -1,5 +1,5 @@ SUBDIR = got-read-blob got-read-commit got-read-object got-read-tree \ got-read-tag got-fetch-pack got-index-pack got-read-pack \ - got-read-gitconfig got-read-gotconfig + got-read-gitconfig got-read-gotconfig got-send-pack .include <bsd.subdir.mk> blob - /dev/null blob + ae3ef0f8e50b2387fd389b1553182be4454aecd9 (mode 644) --- /dev/null +++ libexec/got-send-pack/Makefile @@ -0,0 +1,13 @@ +.PATH:${.CURDIR}/../../lib + +.include "../../got-version.mk" + +PROG= got-send-pack +SRCS= got-send-pack.c error.c inflate.c object_parse.c \ + path.c privsep.c sha1.c + +CPPFLAGS = -I${.CURDIR}/../../include -I${.CURDIR}/../../lib +LDADD = -lutil -lz +DPADD = ${LIBZ} ${LIBUTIL} + +.include <bsd.prog.mk> blob - /dev/null blob + 9e27a52b010a0dc2ae229c241695490181459cca (mode 644) --- /dev/null +++ libexec/got-send-pack/got-send-pack.c @@ -0,0 +1,1038 @@ +/* + * Copyright (c) 2019 Ori Bernstein <ori@openbsd.org> + * Copyright (c) 2021 Stefan Sperling <stsp@openbsd.org> + * + * Permission to use, copy, modify, and distribute this software for any + * purpose with or without fee is hereby granted, provided that the above + * copyright notice and this permission notice appear in all copies. + * + * THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES + * WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF + * MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR + * ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES + * WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN + * ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF + * OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + */ + +#include <sys/types.h> +#include <sys/queue.h> +#include <sys/uio.h> +#include <sys/time.h> +#include <sys/stat.h> + +#include <stdint.h> +#include <errno.h> +#include <imsg.h> +#include <limits.h> +#include <signal.h> +#include <stdio.h> +#include <stdlib.h> +#include <string.h> +#include <ctype.h> +#include <sha1.h> +#include <fcntl.h> +#include <unistd.h> +#include <zlib.h> +#include <err.h> + +#include "got_error.h" +#include "got_object.h" +#include "got_path.h" +#include "got_version.h" +#include "got_fetch.h" +#include "got_reference.h" + +#include "got_lib_sha1.h" +#include "got_lib_delta.h" +#include "got_lib_object.h" +#include "got_lib_object_parse.h" +#include "got_lib_privsep.h" +#include "got_lib_pack.h" + +#ifndef nitems +#define nitems(_a) (sizeof((_a)) / sizeof((_a)[0])) +#endif + +struct got_object *indexed; +static int chattygot; + +static const struct got_error * +readn(ssize_t *off, int fd, void *buf, size_t n) +{ + ssize_t r; + + *off = 0; + while (*off != n) { + r = read(fd, buf + *off, n - *off); + if (r == -1) + return got_error_from_errno("read"); + if (r == 0) + return NULL; + *off += r; + } + return NULL; +} + +static const struct got_error * +flushpkt(int fd) +{ + ssize_t w; + + if (chattygot > 1) + fprintf(stderr, "%s: writepkt: 0000\n", getprogname()); + + w = write(fd, "0000", 4); + if (w == -1) + return got_error_from_errno("write"); + if (w != 4) + return got_error(GOT_ERR_IO); + return NULL; +} + +/* + * Packet header contains a 4-byte hexstring which specifies the length + * of data which follows. + */ +static const struct got_error * +read_pkthdr(int *datalen, int fd) +{ + static const struct got_error *err = NULL; + char lenstr[5]; + long len; + char *e; + int n, i; + ssize_t r; + + *datalen = 0; + + err = readn(&r, fd, lenstr, 4); + if (err) + return err; + if (r == 0) { + /* implicit "0000" */ + if (chattygot > 1) + fprintf(stderr, "%s: readpkt: 0000\n", getprogname()); + return NULL; + } + if (r != 4) + return got_error_msg(GOT_ERR_BAD_PACKET, + "wrong packet header length"); + + lenstr[4] = '\0'; + for (i = 0; i < 4; i++) { + if (!isprint((unsigned char)lenstr[i])) + return got_error_msg(GOT_ERR_BAD_PACKET, + "unprintable character in packet length field"); + } + for (i = 0; i < 4; i++) { + if (!isxdigit((unsigned char)lenstr[i])) { + if (chattygot) + fprintf(stderr, "%s: bad length: '%s'\n", + getprogname(), lenstr); + return got_error_msg(GOT_ERR_BAD_PACKET, + "packet length not specified in hex"); + } + } + errno = 0; + len = strtol(lenstr, &e, 16); + if (lenstr[0] == '\0' || *e != '\0') + return got_error(GOT_ERR_BAD_PACKET); + if (errno == ERANGE && (len == LONG_MAX || len == LONG_MIN)) + return got_error_msg(GOT_ERR_BAD_PACKET, "bad packet length"); + if (len > INT_MAX || len < INT_MIN) + return got_error_msg(GOT_ERR_BAD_PACKET, "bad packet length"); + n = len; + if (n == 0) + return NULL; + if (n <= 4) + return got_error_msg(GOT_ERR_BAD_PACKET, "packet too short"); + n -= 4; + + *datalen = n; + return NULL; +} + +static const struct got_error * +readpkt(int *outlen, int fd, char *buf, int buflen) +{ + const struct got_error *err = NULL; + int datalen, i; + ssize_t n; + + err = read_pkthdr(&datalen, fd); + if (err) + return err; + + if (datalen > buflen) + return got_error(GOT_ERR_NO_SPACE); + + err = readn(&n, fd, buf, datalen); + if (err) + return err; + if (n != datalen) + return got_error_msg(GOT_ERR_BAD_PACKET, "short packet"); + + if (chattygot > 1) { + fprintf(stderr, "%s: readpkt: %zd:\t", getprogname(), n); + for (i = 0; i < n; i++) { + if (isprint(buf[i])) + fputc(buf[i], stderr); + else + fprintf(stderr, "[0x%.2x]", buf[i]); + } + fputc('\n', stderr); + } + + *outlen = n; + return NULL; +} + +static const struct got_error * +writepkt(int fd, char *buf, int nbuf) +{ + char len[5]; + int i; + ssize_t w; + + if (snprintf(len, sizeof(len), "%04x", nbuf + 4) >= sizeof(len)) + return got_error(GOT_ERR_NO_SPACE); + w = write(fd, len, 4); + if (w == -1) + return got_error_from_errno("write"); + if (w != 4) + return got_error(GOT_ERR_IO); + w = write(fd, buf, nbuf); + if (w == -1) + return got_error_from_errno("write"); + if (w != nbuf) + return got_error(GOT_ERR_IO); + if (chattygot > 1) { + fprintf(stderr, "%s: writepkt: %s:\t", getprogname(), len); + for (i = 0; i < nbuf; i++) { + if (isprint(buf[i])) + fputc(buf[i], stderr); + else + fprintf(stderr, "[0x%.2x]", buf[i]); + } + fputc('\n', stderr); + } + return NULL; +} + +static const struct got_error * +tokenize_refline(char **tokens, char *line, int len, int maxtokens) +{ + const struct got_error *err = NULL; + char *p; + size_t i, n = 0; + + for (i = 0; i < maxtokens; i++) + tokens[i] = NULL; + + for (i = 0; n < len && i < maxtokens; i++) { + while (isspace(*line)) { + line++; + n++; + } + p = line; + while (*line != '\0' && + (!isspace(*line) || i == maxtokens - 1)) { + line++; + n++; + } + tokens[i] = strndup(p, line - p); + if (tokens[i] == NULL) { + err = got_error_from_errno("strndup"); + goto done; + } + /* Skip \0 field-delimiter at end of token. */ + while (line[0] == '\0' && n < len) { + line++; + n++; + } + } + if (i <= 2) + err = got_error(GOT_ERR_NOT_REF); +done: + if (err) { + int j; + for (j = 0; j < i; j++) { + free(tokens[j]); + tokens[j] = NULL; + } + } + return err; +} + +static const struct got_error * +parse_refline(char **id_str, char **refname, char **server_capabilities, + char *line, int len) +{ + const struct got_error *err = NULL; + char *tokens[3]; + + err = tokenize_refline(tokens, line, len, nitems(tokens)); + if (err) + return err; + + if (tokens[0]) + *id_str = tokens[0]; + if (tokens[1]) + *refname = tokens[1]; + if (tokens[2]) { + char *p; + *server_capabilities = tokens[2]; + p = strrchr(*server_capabilities, '\n'); + if (p) + *p = '\0'; + } + + return NULL; +} + +#define GOT_CAPA_AGENT "agent" +#define GOT_CAPA_OFS_DELTA "ofs-delta" +#define GOT_CAPA_SIDE_BAND_64K "side-band-64k" +#define GOT_CAPA_REPORT_STATUS "report-status" +#define GOT_CAPA_DELETE_REFS "delete-refs" + +#define GOT_SIDEBAND_PACKFILE_DATA 1 +#define GOT_SIDEBAND_PROGRESS_INFO 2 +#define GOT_SIDEBAND_ERROR_INFO 3 + + +struct got_capability { + const char *key; + const char *value; +}; +static const struct got_capability got_capabilities[] = { + { GOT_CAPA_AGENT, "got/" GOT_VERSION_STR }, + { GOT_CAPA_OFS_DELTA, NULL }, +#if 0 + { GOT_CAPA_SIDE_BAND_64K, NULL }, +#endif + { GOT_CAPA_REPORT_STATUS, NULL }, + { GOT_CAPA_DELETE_REFS, NULL }, +}; + +static const struct got_error * +match_capability(char **my_capabilities, const char *capa, + const struct got_capability *mycapa) +{ + char *equalsign; + char *s; + + equalsign = strchr(capa, '='); + if (equalsign) { + if (strncmp(capa, mycapa->key, equalsign - capa) != 0) + return NULL; + } else { + if (strcmp(capa, mycapa->key) != 0) + return NULL; + } + + if (asprintf(&s, "%s %s%s%s", + *my_capabilities != NULL ? *my_capabilities : "", + mycapa->key, + mycapa->value != NULL ? "=" : "", + mycapa->value != NULL? mycapa->value : "") == -1) + return got_error_from_errno("asprintf"); + + free(*my_capabilities); + *my_capabilities = s; + return NULL; +} + +static const struct got_error * +match_capabilities(char **my_capabilities, char *server_capabilities) +{ + const struct got_error *err = NULL; + char *capa; + size_t i; + + *my_capabilities = NULL; + do { + capa = strsep(&server_capabilities, " "); + for (i = 0; capa != NULL && i < nitems(got_capabilities); i++) { + err = match_capability(my_capabilities, + capa, &got_capabilities[i]); + if (err) + goto done; + } + } while (capa); + + if (*my_capabilities == NULL) { + *my_capabilities = strdup(""); + if (*my_capabilities == NULL) { + err = got_error_from_errno("strdup"); + goto done; + } + } + + /* + * Workaround for github. + * + * Github will accept the pack but fail to update the references + * if we don't have capabilities advertised. Report-status seems + * harmless to add, so we add it. + * + * Github doesn't advertise any capabilities, so we can't check + * for compatibility. We just need to add it blindly. + */ + if (strstr(*my_capabilities, GOT_CAPA_REPORT_STATUS) == NULL) { + char *s; + if (asprintf(&s, "%s %s", *my_capabilities, + GOT_CAPA_REPORT_STATUS) == -1) { + err = got_error_from_errno("asprintf"); + goto done; + } + free(*my_capabilities); + *my_capabilities = s; + } +done: + if (err) { + free(*my_capabilities); + *my_capabilities = NULL; + } + return err; +} + +static const struct got_error * +send_upload_progress(struct imsgbuf *ibuf, off_t bytes) +{ + if (imsg_compose(ibuf, GOT_IMSG_SEND_UPLOAD_PROGRESS, 0, 0, -1, + &bytes, sizeof(bytes)) == -1) + return got_error_from_errno( + "imsg_compose SEND_UPLOAD_PROGRESS"); + + return got_privsep_flush_imsg(ibuf); +} + +static const struct got_error * +send_pack_request(struct imsgbuf *ibuf) +{ + if (imsg_compose(ibuf, GOT_IMSG_SEND_PACK_REQUEST, 0, 0, -1, + NULL, 0) == -1) + return got_error_from_errno("imsg_compose SEND_PACK_REQUEST"); + return got_privsep_flush_imsg(ibuf); +} + +static const struct got_error * +send_done(struct imsgbuf *ibuf) +{ + if (imsg_compose(ibuf, GOT_IMSG_SEND_DONE, 0, 0, -1, NULL, 0) == -1) + return got_error_from_errno("imsg_compose SEND_DONE"); + return got_privsep_flush_imsg(ibuf); +} + +static const struct got_error * +recv_packfd(int *packfd, struct imsgbuf *ibuf) +{ + const struct got_error *err; + struct imsg imsg; + + *packfd = -1; + + err = got_privsep_recv_imsg(&imsg, ibuf, 0); + if (err) + return err; + + if (imsg.hdr.type == GOT_IMSG_STOP) { + err = got_error(GOT_ERR_CANCELLED); + goto done; + } + + if (imsg.hdr.type != GOT_IMSG_SEND_PACKFD) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + goto done; + } + + if (imsg.hdr.len - IMSG_HEADER_SIZE != 0) { + err = got_error(GOT_ERR_PRIVSEP_LEN); + goto done; + } + + *packfd = imsg.fd; +done: + imsg_free(&imsg); + return err; +} + +static const struct got_error * +send_pack_file(int sendfd, int packfd, struct imsgbuf *ibuf) +{ + const struct got_error *err; + unsigned char buf[8192]; + ssize_t r, w; + off_t wtotal = 0; + + if (lseek(packfd, 0L, SEEK_SET) == -1) + return got_error_from_errno("lseek"); + + for (;;) { + r = read(packfd, buf, sizeof(buf)); + if (r == -1) + return got_error_from_errno("read"); + if (r == 0) + break; + w = write(sendfd, buf, r); + if (w == -1) + return got_error_from_errno("write"); + if (w != r) + return got_error(GOT_ERR_IO); + wtotal += w; + err = send_upload_progress(ibuf, wtotal); + if (err) + return err; + } + + return NULL; +} + +static const struct got_error * +send_error(const char *buf, size_t len) +{ + static char msg[1024]; + size_t i; + + for (i = 0; i < len && i < sizeof(msg) - 1; i++) { + if (!isprint(buf[i])) + return got_error_msg(GOT_ERR_BAD_PACKET, + "non-printable error message received from server"); + msg[i] = buf[i]; + } + msg[i] = '\0'; + return got_error_msg(GOT_ERR_SEND_FAILED, msg); +} + +static const struct got_error * +send_their_ref(struct imsgbuf *ibuf, struct got_object_id *refid, + const char *refname) +{ + const struct got_error *err = NULL; + struct ibuf *wbuf; + size_t len, reflen = strlen(refname); + + len = sizeof(struct got_imsg_send_remote_ref) + reflen; + if (len >= MAX_IMSGSIZE - IMSG_HEADER_SIZE) + return got_error(GOT_ERR_NO_SPACE); + + wbuf = imsg_create(ibuf, GOT_IMSG_SEND_REMOTE_REF, 0, 0, len); + if (wbuf == NULL) + return got_error_from_errno("imsg_create SEND_REMOTE_REF"); + + /* Keep in sync with struct got_imsg_send_remote_ref definition! */ + if (imsg_add(wbuf, refid->sha1, SHA1_DIGEST_LENGTH) == -1) { + err = got_error_from_errno("imsg_add SEND_REMOTE_REF"); + ibuf_free(wbuf); + return err; + } + if (imsg_add(wbuf, &reflen, sizeof(reflen)) == -1) { + err = got_error_from_errno("imsg_add SEND_REMOTE_REF"); + ibuf_free(wbuf); + return err; + } + if (imsg_add(wbuf, refname, reflen) == -1) { + err = got_error_from_errno("imsg_add SEND_REMOTE_REF"); + ibuf_free(wbuf); + return err; + } + + wbuf->fd = -1; + imsg_close(ibuf, wbuf); + return got_privsep_flush_imsg(ibuf); +} + +static const struct got_error * +send_ref_status(struct imsgbuf *ibuf, const char *refname, int success, + struct got_pathlist_head *refs, struct got_pathlist_head *delete_refs) + +{ + const struct got_error *err = NULL; + struct ibuf *wbuf; + size_t len, reflen = strlen(refname); + struct got_pathlist_entry *pe; + int ref_valid = 0; + char *eol; + + eol = strchr(refname, '\n'); + if (eol == NULL) { + return got_error_msg(GOT_ERR_BAD_PACKET, + "unexpected message from server"); + } + *eol = '\0'; + + TAILQ_FOREACH(pe, refs, entry) { + if (strcmp(refname, pe->path) == 0) { + ref_valid = 1; + break; + } + } + if (!ref_valid) { + TAILQ_FOREACH(pe, delete_refs, entry) { + if (strcmp(refname, pe->path) == 0) { + ref_valid = 1; + break; + } + } + } + if (!ref_valid) { + return got_error_msg(GOT_ERR_BAD_PACKET, + "unexpected message from server"); + } + + len = sizeof(struct got_imsg_send_ref_status) + reflen; + if (len >= MAX_IMSGSIZE - IMSG_HEADER_SIZE) + return got_error(GOT_ERR_NO_SPACE); + + wbuf = imsg_create(ibuf, GOT_IMSG_SEND_REF_STATUS, + 0, 0, len); + if (wbuf == NULL) + return got_error_from_errno("imsg_create SEND_REF_STATUS"); + + /* Keep in sync with struct got_imsg_send_ref_status definition! */ + if (imsg_add(wbuf, &success, sizeof(success)) == -1) { + err = got_error_from_errno("imsg_add SEND_REF_STATUS"); + ibuf_free(wbuf); + return err; + } + if (imsg_add(wbuf, &reflen, sizeof(reflen)) == -1) { + err = got_error_from_errno("imsg_add SEND_REF_STATUS"); + ibuf_free(wbuf); + return err; + } + if (imsg_add(wbuf, refname, reflen) == -1) { + err = got_error_from_errno("imsg_add SEND_REF_STATUS"); + ibuf_free(wbuf); + return err; + } + + wbuf->fd = -1; + imsg_close(ibuf, wbuf); + return got_privsep_flush_imsg(ibuf); +} + +static const struct got_error * +describe_refchange(int *n, int *sent_my_capabilites, + const char *my_capabilities, char *buf, size_t bufsize, + const char *refname, const char *old_hashstr, const char *new_hashstr) +{ + *n = snprintf(buf, bufsize, "%s %s %s", + old_hashstr, new_hashstr, refname); + if (*n >= bufsize) + return got_error(GOT_ERR_NO_SPACE); + + /* + * We must announce our capabilities along with the first + * reference. Unfortunately, the protocol requires an embedded + * NUL as a separator between reference name and capabilities, + * which we have to deal with here. + * It also requires a linefeed for terminating packet data. + */ + if (!*sent_my_capabilites) { + int m; + if (*n >= bufsize - 1) + return got_error(GOT_ERR_NO_SPACE); + m = snprintf(buf + *n + 1, /* offset after '\0' */ + bufsize - (*n + 1), "%s\n", my_capabilities); + if (*n + m >= bufsize) + return got_error(GOT_ERR_NO_SPACE); + *n += m; + *sent_my_capabilites = 1; + } else { + *n = strlcat(buf, "\n", bufsize); + if (*n >= bufsize) + return got_error(GOT_ERR_NO_SPACE); + } + + return NULL; +} + +static const struct got_error * +send_pack(int fd, struct got_pathlist_head *refs, + struct got_pathlist_head *delete_refs, struct imsgbuf *ibuf) +{ + const struct got_error *err = NULL; + char buf[GOT_FETCH_PKTMAX]; + unsigned char zero_id[SHA1_DIGEST_LENGTH] = { 0 }; + char old_hashstr[SHA1_DIGEST_STRING_LENGTH]; + char new_hashstr[SHA1_DIGEST_STRING_LENGTH]; + struct got_pathlist_head their_refs; + int is_firstpkt = 1; + int n, nsent = 0; + int packfd = -1; + char *id_str = NULL, *refname = NULL; + struct got_object_id *id = NULL; + char *server_capabilities = NULL, *my_capabilities = NULL; + struct got_pathlist_entry *pe; + int sent_my_capabilites = 0; + + TAILQ_INIT(&their_refs); + + if (TAILQ_EMPTY(refs) && TAILQ_EMPTY(delete_refs)) + return got_error(GOT_ERR_SEND_EMPTY); + + while (1) { + err = readpkt(&n, fd, buf, sizeof(buf)); + if (err) + goto done; + if (n == 0) + break; + if (n >= 4 && strncmp(buf, "ERR ", 4) == 0) { + err = send_error(&buf[4], n - 4); + goto done; + } + err = parse_refline(&id_str, &refname, &server_capabilities, + buf, n); + if (err) + goto done; + if (is_firstpkt) { + if (chattygot && server_capabilities[0] != '\0') + fprintf(stderr, "%s: server capabilities: %s\n", + getprogname(), server_capabilities); + err = match_capabilities(&my_capabilities, + server_capabilities); + if (err) + goto done; + if (chattygot) + fprintf(stderr, "%s: my capabilities:%s\n", + getprogname(), my_capabilities); + is_firstpkt = 0; + } + if (strstr(refname, "^{}")) { + if (chattygot) { + fprintf(stderr, "%s: ignoring %s\n", + getprogname(), refname); + } + continue; + } + + id = malloc(sizeof(*id)); + if (id == NULL) { + err = got_error_from_errno("malloc"); + goto done; + } + if (!got_parse_sha1_digest(id->sha1, id_str)) { + err = got_error(GOT_ERR_BAD_OBJ_ID_STR); + goto done; + } + err = send_their_ref(ibuf, id, refname); + if (err) + goto done; + + err = got_pathlist_append(&their_refs, refname, id); + if (chattygot) + fprintf(stderr, "%s: remote has %s %s\n", + getprogname(), refname, id_str); + free(id_str); + id_str = NULL; + refname = NULL; /* do not free; owned by their_refs */ + id = NULL; /* do not free; owned by their_refs */ + } + + if (!TAILQ_EMPTY(delete_refs)) { + if (my_capabilities == NULL || + strstr(my_capabilities, GOT_CAPA_DELETE_REFS) == NULL) { + err = got_error(GOT_ERR_CAPA_DELETE_REFS); + goto done; + } + } + + TAILQ_FOREACH(pe, delete_refs, entry) { + const char *refname = pe->path; + struct got_pathlist_entry *their_pe; + struct got_object_id *their_id = NULL; + + TAILQ_FOREACH(their_pe, &their_refs, entry) { + const char *their_refname = their_pe->path; + if (got_path_cmp(refname, their_refname, + strlen(refname), strlen(their_refname)) == 0) { + their_id = their_pe->data; + break; + } + } + if (their_id == NULL) { + err = got_error_fmt(GOT_ERR_NOT_REF, + "%s does not exist in remote repository", + refname); + goto done; + } + + got_sha1_digest_to_str(their_id->sha1, old_hashstr, + sizeof(old_hashstr)); + got_sha1_digest_to_str(zero_id, new_hashstr, + sizeof(new_hashstr)); + err = describe_refchange(&n, &sent_my_capabilites, + my_capabilities, buf, sizeof(buf), refname, + old_hashstr, new_hashstr); + if (err) + goto done; + err = writepkt(fd, buf, n); + if (err) + goto done; + if (chattygot) { + fprintf(stderr, "%s: deleting %s %s\n", + getprogname(), refname, old_hashstr); + } + nsent++; + } + + TAILQ_FOREACH(pe, refs, entry) { + const char *refname = pe->path; + struct got_object_id *id = pe->data; + struct got_object_id *their_id = NULL; + struct got_pathlist_entry *their_pe; + + TAILQ_FOREACH(their_pe, &their_refs, entry) { + const char *their_refname = their_pe->path; + if (got_path_cmp(refname, their_refname, + strlen(refname), strlen(their_refname)) == 0) { + their_id = their_pe->data; + break; + } + } + if (their_id) { + if (got_object_id_cmp(id, their_id) == 0) { + if (chattygot) { + fprintf(stderr, + "%s: no change for %s\n", + getprogname(), refname); + } + continue; + } + got_sha1_digest_to_str(their_id->sha1, old_hashstr, + sizeof(old_hashstr)); + } else { + got_sha1_digest_to_str(zero_id, old_hashstr, + sizeof(old_hashstr)); + } + got_sha1_digest_to_str(id->sha1, new_hashstr, + sizeof(new_hashstr)); + err = describe_refchange(&n, &sent_my_capabilites, + my_capabilities, buf, sizeof(buf), refname, + old_hashstr, new_hashstr); + if (err) + goto done; + err = writepkt(fd, buf, n); + if (err) + goto done; + if (chattygot) { + if (their_id) { + fprintf(stderr, "%s: updating %s %s -> %s\n", + getprogname(), refname, old_hashstr, + new_hashstr); + } else { + fprintf(stderr, "%s: creating %s %s\n", + getprogname(), refname, new_hashstr); + } + } + nsent++; + } + err = flushpkt(fd); + if (err) + goto done; + + err = send_pack_request(ibuf); + if (err) + goto done; + + err = recv_packfd(&packfd, ibuf); + if (err) + goto done; + + err = send_pack_file(fd, packfd, ibuf); + if (err) + goto done; + + err = readpkt(&n, fd, buf, sizeof(buf)); + if (err) + goto done; + if (n >= 4 && strncmp(buf, "ERR ", 4) == 0) { + err = send_error(&buf[4], n - 4); + goto done; + } else if (n < 10 || strncmp(buf, "unpack ok\n", 10) != 0) { + err = got_error_msg(GOT_ERR_BAD_PACKET, + "unexpected message from server"); + goto done; + } + + while (nsent > 0) { + err = readpkt(&n, fd, buf, sizeof(buf)); + if (err) + goto done; + if (n < 3) { + err = got_error_msg(GOT_ERR_BAD_PACKET, + "unexpected message from server"); + goto done; + } else if (strncmp(buf, "ok ", 3) == 0) { + err = send_ref_status(ibuf, buf + 3, 1, + refs, delete_refs); + if (err) + goto done; + } else if (strncmp(buf, "ng ", 3) == 0) { + err = send_ref_status(ibuf, buf + 3, 0, + refs, delete_refs); + if (err) + goto done; + } else { + err = got_error_msg(GOT_ERR_BAD_PACKET, + "unexpected message from server"); + goto done; + } + nsent--; + } + + err = send_done(ibuf); +done: + TAILQ_FOREACH(pe, &their_refs, entry) { + free((void *)pe->path); + free(pe->data); + } + got_pathlist_free(&their_refs); + free(id_str); + free(id); + free(refname); + free(server_capabilities); + return err; +} + +int +main(int argc, char **argv) +{ + const struct got_error *err = NULL; + int sendfd, i; + struct imsgbuf ibuf; + struct imsg imsg; + struct got_pathlist_head refs; + struct got_pathlist_head delete_refs; + struct got_pathlist_entry *pe; + struct got_imsg_send_request send_req; + struct got_imsg_send_ref href; + size_t datalen; +#if 0 + static int attached; + while (!attached) + sleep (1); +#endif + + TAILQ_INIT(&refs); + TAILQ_INIT(&delete_refs); + + imsg_init(&ibuf, GOT_IMSG_FD_CHILD); +#ifndef PROFILE + /* revoke access to most system calls */ + if (pledge("stdio recvfd", NULL) == -1) { + err = got_error_from_errno("pledge"); + got_privsep_send_error(&ibuf, err); + return 1; + } +#endif + if ((err = got_privsep_recv_imsg(&imsg, &ibuf, 0)) != 0) { + if (err->code == GOT_ERR_PRIVSEP_PIPE) + err = NULL; + goto done; + } + if (imsg.hdr.type == GOT_IMSG_STOP) + goto done; + if (imsg.hdr.type != GOT_IMSG_SEND_REQUEST) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + goto done; + } + datalen = imsg.hdr.len - IMSG_HEADER_SIZE; + if (datalen < sizeof(send_req)) { + err = got_error(GOT_ERR_PRIVSEP_LEN); + goto done; + } + memcpy(&send_req, imsg.data, sizeof(send_req)); + sendfd = imsg.fd; + imsg_free(&imsg); + + if (send_req.verbosity > 0) + chattygot += send_req.verbosity; + + for (i = 0; i < send_req.nrefs; i++) { + struct got_object_id *id; + char *refname; + + if ((err = got_privsep_recv_imsg(&imsg, &ibuf, 0)) != 0) { + if (err->code == GOT_ERR_PRIVSEP_PIPE) + err = NULL; + goto done; + } + if (imsg.hdr.type == GOT_IMSG_STOP) + goto done; + if (imsg.hdr.type != GOT_IMSG_SEND_REF) { + err = got_error(GOT_ERR_PRIVSEP_MSG); + goto done; + } + datalen = imsg.hdr.len - IMSG_HEADER_SIZE; + if (datalen < sizeof(href)) { + err = got_error(GOT_ERR_PRIVSEP_LEN); + goto done; + } + memcpy(&href, imsg.data, sizeof(href)); + if (datalen - sizeof(href) < href.name_len) { + err = got_error(GOT_ERR_PRIVSEP_LEN); + goto done; + } + refname = malloc(href.name_len + 1); + if (refname == NULL) { + err = got_error_from_errno("malloc"); + goto done; + } + memcpy(refname, imsg.data + sizeof(href), href.name_len); + refname[href.name_len] = '\0'; + + /* + * Prevent sending of references that won't make any + * sense outside the local repository's context. + */ + if (strncmp(refname, "refs/got/", 9) == 0 || + strncmp(refname, "refs/remotes/", 13) == 0) { + err = got_error_fmt(GOT_ERR_SEND_BAD_REF, + "%s", refname); + goto done; + } + + id = malloc(sizeof(*id)); + if (id == NULL) { + free(refname); + err = got_error_from_errno("malloc"); + goto done; + } + memcpy(id->sha1, href.id, SHA1_DIGEST_LENGTH); + if (href.delete) + err = got_pathlist_append(&delete_refs, refname, id); + else + err = got_pathlist_append(&refs, refname, id); + if (err) { + free(refname); + free(id); + goto done; + } + + imsg_free(&imsg); + } + + err = send_pack(sendfd, &refs, &delete_refs, &ibuf); +done: + TAILQ_FOREACH(pe, &refs, entry) { + free((char *)pe->path); + free(pe->data); + } + got_pathlist_free(&refs); + TAILQ_FOREACH(pe, &delete_refs, entry) { + free((char *)pe->path); + free(pe->data); + } + got_pathlist_free(&delete_refs); + if (sendfd != -1 && close(sendfd) == -1 && err == NULL) + err = got_error_from_errno("close"); + if (err != NULL && err->code != GOT_ERR_CANCELLED) { + fprintf(stderr, "%s: %s\n", getprogname(), err->msg); + got_privsep_send_error(&ibuf, err); + } + + exit(0); +} blob - ef0efe1dbbe320c62fff14e7f2e0b94db43f38ff blob + 68314f1e77bb35025aeb5401ca14f24b773350a5 --- regress/cmdline/Makefile +++ regress/cmdline/Makefile @@ -77,6 +77,9 @@ clone: fetch: ./fetch.sh -q -r "$(GOT_TEST_ROOT)" +send: + ./send.sh -q -r "$(GOT_TEST_ROOT)" + tree: ./tree.sh -q -r "$(GOT_TEST_ROOT)" blob - /dev/null blob + 9ac2cfc3c6e7719719f8547f7f5a1de2ece46017 (mode 755) --- /dev/null +++ regress/cmdline/send.sh @@ -0,0 +1,1042 @@ +#!/bin/sh +# +# Copyright (c) 2021 Stefan Sperling <stsp@openbsd.org> +# +# Permission to use, copy, modify, and distribute this software for any +# purpose with or without fee is hereby granted, provided that the above +# copyright notice and this permission notice appear in all copies. +# +# THE SOFTWARE IS PROVIDED "AS IS" AND THE AUTHOR DISCLAIMS ALL WARRANTIES +# WITH REGARD TO THIS SOFTWARE INCLUDING ALL IMPLIED WARRANTIES OF +# MERCHANTABILITY AND FITNESS. IN NO EVENT SHALL THE AUTHOR BE LIABLE FOR +# ANY SPECIAL, DIRECT, INDIRECT, OR CONSEQUENTIAL DAMAGES OR ANY DAMAGES +# WHATSOEVER RESULTING FROM LOSS OF USE, DATA OR PROFITS, WHETHER IN AN +# ACTION OF CONTRACT, NEGLIGENCE OR OTHER TORTIOUS ACTION, ARISING OUT OF +# OR IN CONNECTION WITH THE USE OR PERFORMANCE OF THIS SOFTWARE. + +. ./common.sh + +test_send_basic() { + local testroot=`test_init send_basic` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + got clone -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + cat > $testroot/repo/.git/got.conf <<EOF +remote "origin" { + protocol ssh + server 127.0.0.1 + repository "$testroot/repo-clone" +} +EOF + got tag -r $testroot/repo -m '1.0' 1.0 >/dev/null + tag_id=`got ref -r $testroot/repo -l | grep "^refs/tags/1.0" \ + | tr -d ' ' | cut -d: -f2` + + echo "modified alpha" > $testroot/repo/alpha + git_commit $testroot/repo -m "modified alpha" + local commit_id2=`git_show_head $testroot/repo` + + got send -q -r $testroot/repo > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id2" \ + >> $testroot/stdout.expected + echo "refs/tags/1.0: $tag_id" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/HEAD: refs/remotes/origin/master" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got send -r $testroot/repo > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo 'Connecting to "origin" 127.0.0.1' > $testroot/stdout.expected + echo "Already up-to-date" >> $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + +test_send_rebase_required() { + local testroot=`test_init send_rebase_required` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + got clone -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + cat > $testroot/repo/.git/got.conf <<EOF +remote "origin" { + protocol ssh + server 127.0.0.1 + repository "$testroot/repo-clone" +} +EOF + echo "modified alpha" > $testroot/repo/alpha + git_commit $testroot/repo -m "modified alpha" + local commit_id2=`git_show_head $testroot/repo` + + got checkout $testroot/repo-clone $testroot/wt-clone >/dev/null + echo "modified alpha, too" > $testroot/wt-clone/alpha + (cd $testroot/wt-clone && got commit -m 'change alpha' >/dev/null) + + got send -q -r $testroot/repo > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + echo "got: refs/heads/master: fetch and rebase required" \ + > $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + fi + test_done "$testroot" "$ret" +} + +test_send_rebase_required_overwrite() { + local testroot=`test_init send_rebase_required_overwrite` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + got clone -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + cat > $testroot/repo/.git/got.conf <<EOF +remote "foobar" { + protocol ssh + server 127.0.0.1 + repository "$testroot/repo-clone" +} +EOF + echo "modified alpha" > $testroot/repo/alpha + git_commit $testroot/repo -m "modified alpha" + local commit_id2=`git_show_head $testroot/repo` + + got checkout $testroot/repo-clone $testroot/wt-clone >/dev/null + echo "modified alpha, too" > $testroot/wt-clone/alpha + (cd $testroot/wt-clone && got commit -m 'change alpha' >/dev/null) + local commit_id3=`git_show_head $testroot/repo-clone` + + # non-default remote requires an explicit argument + got send -q -r $testroot/repo -f > $testroot/stdout \ + 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + echo "got: origin: remote repository not found" \ + > $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + test_done "$testroot" "$ret" + return 1 + fi + + got send -q -r $testroot/repo -f foobar > $testroot/stdout \ + 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/foobar/master: $commit_id2" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + wt_uuid=`(cd $testroot/wt-clone && got info | grep 'UUID:' | \ + cut -d ':' -f 2 | tr -d ' ')` + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/got/worktree/base-$wt_uuid: $commit_id3" \ + >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/HEAD: refs/remotes/origin/master" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + +test_send_delete() { + local testroot=`test_init send_delete` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + # branch1 exists in both repositories + got branch -r $testroot/repo branch1 + + got clone -a -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + cat > $testroot/repo/.git/got.conf <<EOF +remote "origin" { + protocol ssh + server 127.0.0.1 + repository "$testroot/repo-clone" +} +EOF + # branch2 exists only in the remote repository + got branch -r $testroot/repo-clone branch2 + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/branch1: $commit_id" >> $testroot/stdout.expected + echo "refs/heads/branch2: $commit_id" >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id" >> $testroot/stdout.expected + + # Sending changes for a branch and deleting it at the same + # time is not allowed. + got send -q -r $testroot/repo -d branch1 -b branch1 \ + > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + echo -n "got: changes on refs/heads/branch1 will be sent to server" \ + > $testroot/stderr.expected + echo ": reference cannot be deleted" >> $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + test_done "$testroot" "$ret" + return 1 + fi + + got send -q -r $testroot/repo -d refs/heads/branch1 origin \ + > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + got send -q -r $testroot/repo -d refs/heads/branch2 origin \ + > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + # branchX exists in neither repository + got send -q -r $testroot/repo -d refs/heads/branchX origin \ + > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + echo -n "got-send-pack: refs/heads/branchX does not exist in remote " \ + > $testroot/stderr.expected + echo "repository: no such reference found" >> $testroot/stderr.expected + echo "got: no such reference found" >> $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + test_done "$testroot" "$ret" + return 1 + fi + + # References outside of refs/heads/ cannot be deleted with 'got send'. + got send -q -r $testroot/repo -d refs/tags/1.0 origin \ + > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + echo -n "got-send-pack: refs/heads/refs/tags/1.0 does not exist " \ + > $testroot/stderr.expected + echo "in remote repository: no such reference found" \ + >> $testroot/stderr.expected + echo "got: no such reference found" >> $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/branch1: $commit_id" >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id" >> $testroot/stdout.expected + echo "refs/remotes/origin/HEAD: refs/remotes/origin/master" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/branch1: $commit_id" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + +test_send_clone_and_send() { + local testroot=`test_init send_clone_and_send` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + (cd $testroot/repo && git config receive.denyCurrentBranch ignore) + + got clone -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + got checkout $testroot/repo-clone $testroot/wt >/dev/null + echo "modified alpha" > $testroot/wt/alpha + (cd $testroot/wt && got commit -m "modified alpha" >/dev/null) + local commit_id2=`git_show_head $testroot/repo-clone` + + (cd $testroot/wt && got send -q > $testroot/stdout 2> $testroot/stderr) + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + wt_uuid=`(cd $testroot/wt && got info | grep 'UUID:' | \ + cut -d ':' -f 2 | tr -d ' ')` + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/got/worktree/base-$wt_uuid: $commit_id2" \ + >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/HEAD: refs/remotes/origin/master" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id2" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + +test_send_tags() { + local testroot=`test_init send_tags` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + got clone -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + cat > $testroot/repo/.git/got.conf <<EOF +remote "origin" { + protocol ssh + server 127.0.0.1 + repository "$testroot/repo-clone" +} +EOF + got tag -r $testroot/repo -m '1.0' 1.0 >/dev/null + tag_id=`got ref -r $testroot/repo -l | grep "^refs/tags/1.0" \ + | tr -d ' ' | cut -d: -f2` + + echo "modified alpha" > $testroot/repo/alpha + git_commit $testroot/repo -m "modified alpha" + local commit_id2=`git_show_head $testroot/repo` + + got tag -r $testroot/repo -m '2.0' 2.0 >/dev/null + tag_id2=`got ref -r $testroot/repo -l | grep "^refs/tags/2.0" \ + | tr -d ' ' | cut -d: -f2` + + got send -q -r $testroot/repo -T > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id2" \ + >> $testroot/stdout.expected + echo "refs/tags/1.0: $tag_id" >> $testroot/stdout.expected + echo "refs/tags/2.0: $tag_id2" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/HEAD: refs/remotes/origin/master" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id" \ + >> $testroot/stdout.expected + echo "refs/tags/1.0: $tag_id" >> $testroot/stdout.expected + echo "refs/tags/2.0: $tag_id2" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got tag -l -r $testroot/repo-clone | grep ^tag | sort > $testroot/stdout + echo "tag 1.0 $tag_id" > $testroot/stdout.expected + echo "tag 2.0 $tag_id2" >> $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + # Overwriting an existing tag 'got send -f'. + got ref -r $testroot/repo -d refs/tags/1.0 >/dev/null + got tag -r $testroot/repo -m '1.0' 1.0 >/dev/null + tag_id3=`got ref -r $testroot/repo -l | grep "^refs/tags/1.0" \ + | tr -d ' ' | cut -d: -f2` + + got send -q -r $testroot/repo -t 1.0 > $testroot/stdout \ + 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "got: refs/tags/1.0: tag already exists on server" \ + > $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + test_done "$testroot" "$ret" + return 1 + fi + + # attempting the same with -T should fail, too + got send -q -r $testroot/repo -T > $testroot/stdout \ + 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "got: refs/tags/1.0: tag already exists on server" \ + > $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + test_done "$testroot" "$ret" + return 1 + fi + + got tag -l -r $testroot/repo-clone | grep ^tag | sort > $testroot/stdout + echo "tag 1.0 $tag_id" > $testroot/stdout.expected + echo "tag 2.0 $tag_id2" >> $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + # overwrite the 1.0 tag only + got send -q -r $testroot/repo -t 1.0 -f > $testroot/stdout \ + 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + got tag -l -r $testroot/repo-clone | grep ^tag | sort > $testroot/stdout + echo "tag 1.0 $tag_id3" > $testroot/stdout.expected + echo "tag 2.0 $tag_id2" >> $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + +test_send_new_branch() { + local testroot=`test_init send_new_branch` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + (cd $testroot/repo && git config receive.denyCurrentBranch ignore) + + got clone -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + got branch -r $testroot/repo-clone foo >/dev/null + got checkout -b foo $testroot/repo-clone $testroot/wt >/dev/null + echo "modified alpha" > $testroot/wt/alpha + (cd $testroot/wt && got commit -m "modified alpha" >/dev/null) + local commit_id2=`git_show_branch_head $testroot/repo-clone foo` + + (cd $testroot/wt && got send -q > $testroot/stdout 2> $testroot/stderr) + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/foo: $commit_id2" >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + wt_uuid=`(cd $testroot/wt && got info | grep 'UUID:' | \ + cut -d ':' -f 2 | tr -d ' ')` + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/got/worktree/base-$wt_uuid: $commit_id2" \ + >> $testroot/stdout.expected + echo "refs/heads/foo: $commit_id2" >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id" >> $testroot/stdout.expected + echo "refs/remotes/origin/HEAD: refs/remotes/origin/master" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/foo: $commit_id2" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + +test_send_all_branches() { + local testroot=`test_init send_all_branches` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + (cd $testroot/repo && git config receive.denyCurrentBranch ignore) + + got clone -q $testurl/repo $testroot/repo-clone + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + got checkout $testroot/repo-clone $testroot/wt >/dev/null + echo "modified alpha" > $testroot/wt/alpha + (cd $testroot/wt && got commit -m "modified alpha" >/dev/null) + local commit_id2=`git_show_head $testroot/repo-clone` + + got branch -r $testroot/repo-clone foo >/dev/null + (cd $testroot/wt && got update -b foo >/dev/null) + echo "modified beta on new branch foo" > $testroot/wt/beta + (cd $testroot/wt && got commit -m "modified beta" >/dev/null) + local commit_id3=`git_show_branch_head $testroot/repo-clone foo` + + got branch -r $testroot/repo-clone bar >/dev/null + (cd $testroot/wt && got update -b bar >/dev/null) + echo "modified beta again on new branch bar" > $testroot/wt/beta + (cd $testroot/wt && got commit -m "modified beta" >/dev/null) + local commit_id4=`git_show_branch_head $testroot/repo-clone bar` + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/bar: $commit_id4" >> $testroot/stdout.expected + echo "refs/heads/foo: $commit_id3" >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + + got send -a -q -r $testroot/repo-clone -b master > $testroot/stdout \ + 2> $testroot/stderr + ret="$?" + if [ "$ret" == "0" ]; then + echo "got send command succeeded unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + echo "got: -a and -b options are mutually exclusive" \ + > $testroot/stderr.expected + cmp -s $testroot/stderr $testroot/stderr.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stderr.expected $testroot/stderr + test_done "$testroot" "$ret" + return 1 + fi + + got send -a -q -r $testroot/repo-clone > $testroot/stdout \ + 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/bar: $commit_id4" >> $testroot/stdout.expected + echo "refs/heads/foo: $commit_id3" >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo-clone > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + wt_uuid=`(cd $testroot/wt && got info | grep 'UUID:' | \ + cut -d ':' -f 2 | tr -d ' ')` + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/got/worktree/base-$wt_uuid: $commit_id4" \ + >> $testroot/stdout.expected + echo "refs/heads/bar: $commit_id4" >> $testroot/stdout.expected + echo "refs/heads/foo: $commit_id3" >> $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/HEAD: refs/remotes/origin/master" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/bar: $commit_id4" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/foo: $commit_id3" \ + >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id2" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + +test_send_to_empty_repo() { + local testroot=`test_init send_to_empty_repo` + local testurl=ssh://127.0.0.1/$testroot + local commit_id=`git_show_head $testroot/repo` + + got init $testroot/repo2 + + ret="$?" + if [ "$ret" != "0" ]; then + echo "got clone command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + cat > $testroot/repo/.git/got.conf <<EOF +remote "origin" { + protocol ssh + server 127.0.0.1 + repository "$testroot/repo2" +} +EOF + echo "modified alpha" > $testroot/repo/alpha + git_commit $testroot/repo -m "modified alpha" + local commit_id2=`git_show_head $testroot/repo` + + got send -q -r $testroot/repo > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo -n > $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + # XXX Workaround: We cannot give the target for HEAD to 'got init' + got ref -r $testroot/repo2 -s refs/heads/master HEAD + + got ref -l -r $testroot/repo > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + echo "refs/remotes/origin/master: $commit_id2" \ + >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got ref -l -r $testroot/repo2 > $testroot/stdout + if [ "$ret" != "0" ]; then + echo "got ref command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo "HEAD: refs/heads/master" > $testroot/stdout.expected + echo "refs/heads/master: $commit_id2" >> $testroot/stdout.expected + + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + test_done "$testroot" "$ret" + return 1 + fi + + got send -r $testroot/repo > $testroot/stdout 2> $testroot/stderr + ret="$?" + if [ "$ret" != "0" ]; then + echo "got send command failed unexpectedly" >&2 + test_done "$testroot" "$ret" + return 1 + fi + + echo 'Connecting to "origin" 127.0.0.1' > $testroot/stdout.expected + echo "Already up-to-date" >> $testroot/stdout.expected + cmp -s $testroot/stdout $testroot/stdout.expected + ret="$?" + if [ "$ret" != "0" ]; then + diff -u $testroot/stdout.expected $testroot/stdout + fi + test_done "$testroot" "$ret" +} + + +test_parseargs "$@" +run_test test_send_basic +run_test test_send_rebase_required +run_test test_send_rebase_required_overwrite +run_test test_send_delete +run_test test_send_clone_and_send +run_test test_send_tags +run_test test_send_new_branch +run_test test_send_all_branches +run_test test_send_to_empty_repo

2021-08-25 19:10 Stefan Sperling:
add 'got send' command
- 2021-08-26 07:17 Stefan Sperling:
  add 'got send' command
- 2021-08-26 13:14 Christian Weisgerber:
  add 'got send' command