Working with the meta repository - OpenSlides/OpenSlides GitHub Wiki

Multiple services use the meta repository as a submodule. This page summarizes a few tips & tricks on working with the meta repo and submodules in general.

Note: It is assumed in all given scripts (and generally encouraged) that you follow the naming convention of using upstream as the name for the OpenSlides remote.

1. Create a PR in the meta repo with the changes.
2. Adjust the service(s) which you are willing/able to.
3. Wait for all other services to implement the changes (if they need changes at all).
   • Note: Thanks to the meta repo, this has become as easy as checking out the PR in the submodule.
4. Merge all service PRs, then afterwards merge the meta PR.
   • This order prevents the existence of a short time window where you do a git merge main in another PR and then recursivly merge the meta repo, leading to a situation where the meta changes are already included in the PR, but the respective service changes aren't yet, therefore failing in the CI and needing another git merge main.
5. The meta repo will auomatically create PRs in all relevant repos with the newest commit. Merge them as well to have all repos synchronous and up-to-date.
6. Done! Thanks to the "merge commit" strategy in the meta repo, it is not necessary to update any services as the commit hash still exists in the main branch.

Merge conflicts regarding the meta repo occur often since GitHub cannot handle more than very simple forward merges in submodules. Therefore, if the meta repo gets updated on the main branch, but the PR branch's version of the meta repo does not yet have these commits, it will lead to a merge conflict. Thankfully, this is relatively easy to solve. In the following, the full process of merging an updated main branch into some other branch is described:

git fetch upstream main             # fetch updated content from main
git checkout <branch>               # checkout the desired branch
git merge upstream/main             # will lead to a conflict in the meta repo's folder
pushd .git/modules/openslides-meta  # the git folder of the meta repo always lies here, independently of the actual location in the repo
git checkout <meta-branch>          # checkout the branch of the meta repo where the respective changes for this feature are
git fetch upstream main             # fetch updated content from the meta repo's main branch
git merge upstream/main             # merge the meta repo main - most of the time, this should work without conflicts
# if any conflicts occur, resolve them and finish the merge
git push                            # push the meta repo changes
popd                                # go back to service repo folder
# resolve other merge conflicts, if necessary
git add .                           # add all resolved conflicts (including the meta repo) to the index
git merge --continue                # finish the merge
git push                            # push the changes

The middle part of updating the meta repo can easily be extracted into a script:

#!/bin/bash

set -e

pushd .git/modules/openslides-meta
git checkout $1
git fetch upstream main
git merge --no-edit upstream/main  # added --no-edit option to skip the editor
git push
popd

If you save this script somewhere under, e.g., merge-meta.sh, you can call it with merge-meta.sh <meta-branch-name> to automatically update the meta repo in any service. For even easier access, add it to your .bashrc with an alias: alias merge-meta=/path/to/folder/merge-meta.sh. Only downside is that autocomplete will not work for this script, but this would be too much work right now to implement.

If the script terminates with a message a la Failed to merge submodule xy, there are merge conflicts inside the meta repo which need manual resolving. Go into the meta repo folder, resolve the conflicts and finish the merge with git add . && git merge --continue. Otherwise, the merge was successful and you can directly continue the merge in the service repo.

When working with git repositories and especially when working with submodules, a git aware prompt comes in very handy. One candidate is https://github.com/jimeh/git-aware-prompt, where you can find installation instructions in the README. If you follow these instructions, the current branch name will be displayed in the console after the current working directory. If you have any local changes, an asterics is displayed afterwards. This is important when working with submodules as, e.g., after a git stash in a service repo, you might notice that the asterics is still there if the meta submodule is still checked out to some other commit. You first have to do a git submodule update (assuming you already pushed your local meta repo commits) to reset it to the desired commit and remove the asterics. Only then can you check out another branch.

git submodule add --name openslides-meta -- [email protected]:OpenSlides/openslides-meta.git path/in/repo

Replace path/in/repo with the actual path where the meta folder should live. It does not have to be named openslides-meta, most common is simply meta. Importantly, for uniformity, the name should always be openslides-meta, regardless of the path.

In a freshly cloned repo (not only the meta repo), the remotes are probably setup wrong. The most common idiom when working with forks is to name your personal remote origin and the OpenSlides remote upstream. The following script sets up these remotes:

user=<your github username>
repo=$(basename `git config --get remote.origin.url`)

git remote set-url origin [email protected]:$user/$repo
git remote add upstream [email protected]:OpenSlides/$repo

Other remotes, e.g., from other developers can be added analogous to upstream.

Per default, git does not checkout the respective submodule version when switching to another branch. However, most of the time, this is necessary, since the actual code is tied to, e.g., the content of the models.yml. There are two ways to do that:

• use git checkout --recurse-submodules <branch> on every checkout
• set git config --global submodule.recurse true once

The latter option means less work and not having to think about it, so it is the recommended way of doing this.

If you have backend branches from before the introduction of the meta repo, it is a bit tricky to merge the new main into them to have them up-to-date. Here is a guideline on how to do this:

git checkout <branch-to-update>
git fetch upstream
git merge upstream/main
rm -r global/meta
rm -r global/meta*
# remove all other conflicts, as far as possible
git add .
git submodule add --name openslides-meta --force -- https://github.com/OpenSlides/openslides-meta.git global/meta
git submodule update --init
cd global/meta
git checkout main  # or whichever branch you need
git pull
cd ../..
# remove remaining conflicts if they need the new models yaml
git add .
git merge --continue

There may be some caveats which may lead to this not working 100%, so additional commands in between may be necessary.

The following command returns the relative path of the openslides-meta submodule inside a service repository:

git config --file .gitmodules --get submodule.openslides-meta.path