From: Omar Polo <op@omarpolo.com>
Subject: Re: i386 / got-read-pack: imsg_add TREE_ENTRY: Result too large
To: Sebastien Marie <semarie@online.fr>
Cc: gameoftrees@openbsd.org
Date: Sat, 02 Jul 2022 12:40:32 +0200

Sebastien Marie <semarie@online.fr> wrote:
> Hi,
> 
> Currently, on OpenBSD i386 -current (-current as in OpenBSD 7.1-current 
> (GENERIC.MP) #274: Sun Jun 26 17:11:24 MDT 2022), I have an error while doing 
> 'got up' (on existing checkout) or 'got checkout' (to create a new checkout).
> 
> Initially, the problem shown on existing checkout. I tried to fsck/gc the 
> repository to ensure it was in sane state, and I finally tried with a fresh 
> clone. It is still the same behaviour.
> 
> I don't have the problem on amd64, so it might be a integer size problem.
> 
> Steps to reproduce:
> 
> $ arch
> OpenBSD.i386
> 
> $ git clone --mirror https://github.com/openbsd/ports ports-new.git
> Clonage dans le dépôt nu 'ports-new.git'
> remote: Enumerating objects: 1734457, done.
> remote: Counting objects: 100% (1421/1421), done.
> remote: Compressing objects: 100% (1101/1101), done.
> remote: Total 1734457 (delta 332), reused 1301 (delta 303), pack-reused 1733036
> Réception d'objets: 100% (1734457/1734457), 461.01 Mio | 1.58 Mio/s, fait.
> Résolution des deltas: 100% (1127907/1127907), fait.
> 
> $ got --version
> got 0.72
> 
> $ got checkout ports-new.git
> [...]
> A  /home/semarie/repos/openbsd/ports-new/databases/xapian-core/Makefile
> A  /home/semarie/repos/openbsd/ports-new/databases/xapian-core/distinfo
> A  /home/semarie/repos/openbsd/ports-new/databases/xapian-core/patches/patch-cmake_xapian-config_cmake_in
> A  /home/semarie/repos/openbsd/ports-new/databases/xapian-core/pkg/DESCR
> A  /home/semarie/repos/openbsd/ports-new/databases/xapian-core/pkg/PLIST
> got: Result too large
> got-read-pack: imsg_add TREE_ENTRY: Result too large
> 
> $ echo $?
> 1
> 
> $
> 
> Thanks.

I think I got it.  In send_tree_entries (lib/privsep.c) we're computing
the size of the got_parsed_tree_entry (pte) that we're about to send as

	size_t len = sizeof(*pte) + pte->namelen;

but it's wrong!  The struct is defined as: (lib/got_lib_object_parse.h)

	struct got_parsed_tree_entry {
		const char *name; /* Points to name in parsed buffer */
		size_t namelen; /* strlen(name) */
		mode_t mode; /* Mode parsed from tree buffer. */
		uint8_t *id; /* Points to ID in parsed tree buffer. */
	};

and we serialize it in send_tree_entries_batch by writing the id (20
bytes), mode, namelen and name in the wbuf.

On amd64 the size of the two pointers hides the issue but on i386 this
error in computing the size of the batch means we try to add to an imsg
more data than what allocated.

The following diff seems to fix the issue for me, `got co' proceeds past
databases/.  (i'm currently at devel/p5-*, this machine is slow)


Cheers,

Omar Polo

diff -s /home/op/w/got
commit - db0dfdd7e5c2c5a38ed7c3291a0615132bcb5945
path + /home/op/w/got (staged changes)
blob - 70eb167c5ee71b29c045bdae0de5c7b7859403f7
blob + 31bc523dadb98ca90e8fcdfc390e7beba909f667
--- lib/privsep.c
+++ lib/privsep.c
@@ -1471,7 +1471,8 @@ send_tree_entries(struct imsgbuf *ibuf, struct got_par
 	i = 0;
 	for (j = 0; j < nentries; j++) {
 		struct got_parsed_tree_entry *pte = &entries[j];
-		size_t len = sizeof(*pte) + pte->namelen;
+		size_t len = SHA1_DIGEST_LENGTH + sizeof(pte->mode) +
+		    sizeof(pte->namelen) + pte->namelen;
 
 		if (j > 0 &&
 		    entries_len + len > MAX_IMSGSIZE - IMSG_HEADER_SIZE) {