From: ori@eigenstate.org
Subject: Re: submodules: got: bad file type
To: stsp@stsp.name, ori@eigenstate.org
Cc: semarie@online.fr, gameoftrees@openbsd.org
Date: Tue, 12 May 2020 08:04:44 -0700

> On Sun, May 10, 2020 at 08:31:26AM -0700, ori@eigenstate.org wrote:
>> The solution I settled on in git9, which I'm fairly happy
>> with: *submodule links* are read-only, but as long as you
>> don't modify them, committing works just fine.
>  
> Interesting.
> 
> How can you tell whether a submodule was modified?
> I assume you detect the cases where the submodule tree entry no longer
> points to the same SHA1 hash ID as it did in the parent tree, or when
> the submodule tree entry is no longer present?
> 
> What does your implementation do when a new submodule tree entry was added?
> Since you call your approach "read-only", I assume such a commit would be
> rejected?

We have no way of adding submodule trees while keeping them as submodules,
so that's not a problem. When checking out commmits, submodule trees are
treated as empty directories ("clone something there yourself, if you want").

When creating a commit, if submodule reference exists and the commit touches
it (ie, we're not just carrying it forward from the previous commit) then the
commit errors out:

	"modifying modules unsupported: <path>"

>> It's easy enough to carry them forward unmodified in tree
>> objects when creating new commits.
> 
> Yes. That is indeed something we could easily allow.
> 
> My question is if that helps a particular use case, or just provides an
> illusion that the commit worked fine. Perhaps, in some unlikely case, we
> create a commit that Git will end up complaining about? Or create a
> condition which some other part of Got will not handle gracefully (which
> I believe is far more likely)?

Sure -- the use case it allows you to work with repositories that contain
submodules, as long as you don't need to change the submodules themselves.
For example:

	repo/
		Makefile
		foo.c
		bar.c
		submodule/


As long as I'm modifying only Makefile, foo.c, or bar.c, then I can make
whatever changes I need to. The submodule itself would be untouchable.

When making a commit to foo.c, I carry forward all of the elements of
the tree except for foo.c. The submodule remains untouched -- as long
as git was happy with it when the submodule was created, I see no
reason for it to become unhappy with it.

> It is another type of node in the repository tree, in addition to files and
> directories. So, from a full system design point of view, allowing users
> to create, modify, and/or delete submodules may mean that we must handle
> this type of tree entry in more places than just when creating commits.

What I'm suggesting is that the submodules are *not* created,
modified, or deleted. They're merely preserved. The only issue
that this approach causes, I think, is pressure to one day
support submodules fully.

I think this covers the majority of uses we'd need in got, where
we ended up stuck with a repository that uses submodules, and
now we need to make some change in it.
 
> For instance, submodule tree entries may end up clashing with a file or
> a directory occupying the same path, during update, rebase, histedit,
> backout, and cherrypick. Or clash with files and directories on disk.
> We have no code to handle such collisions right now, let alone tests for
> such cases, or even basic submodule addition/deletion/modification tests.

Yes. This is simply be a commit that can't be done, and you'd need
to fall back to git.

> With each type of node we allow as a tree entry, our conflict and test
> matrix grows by one dimension, and this creates more work for us.
> For a feature that, I presume, nobody will ever need to work on OpenBSD code.
> The same applies to symlinks, which cannot be committed either right now.
> 
> Of course, such problems could already happen today when Got and Git are
> used on the same repository. I am assuming a project which starts from
> scratch with Got, perhaps importing history from another version control
> system, and adapting workflows accordingly if Git compatibility is required.

Yes. That's a valid concern. I can't say for sure if being able to commit in
the presence of submodules is worth the tradeoff, but I've used it a couple
of times.