Collaborative Git Workflow - ATTPC/ATTPCROOTv2 GitHub Wiki
The purpose of this guide is to provide developers, including those newer to using git for collaborative work, a guide for using git collaboratively. This is adapted from the guide of Anar Manafov for git workflows for development teams. The golden rule of collaborative git development is:
This guide is split into three main sections. First is an overview of the workflow, the second is actual git commands for common tasks in the workflow for developers, third is actual git commands for common tasks in the workflow for managers.
Table of Contents
- remote: A repository on GitHub
- upstream: The main repository ATTPC/ATTPCROOTv2
- origin: You're fork of the main repository
- branch: A branch is labeled by its remote, and branch name. For example upstream/develop, origin/develop, origin/featureXXX
- Feature/Fixes: patches that can wait until the next release. They follow the workflow: private origin/feature -> public upstream/develop -> public upstream/main (i.e. release)
- Hot Fixes: patches that cannot wait until the next release and must be applied ASAP. They follow the workflow: private origin/feature from upstream/main -> upstream/main
- Use one branch per feature/bug and then use a pull request to merge into upstream/develop.
- Only managers are allowed to work on develop/main branches (described below).
- Avoid cherry picking, almost always the same effect can be achieved with good branch management.
- Avoid committing any files automatically generated by the code or large binary/root files (.gitignore is your friend)
- Always rebase your feature/bug branch before merging into upstream/develop
- Always specify the remote and branch when pushing
- Use
git pull --rebaseto avoid merges from upstream commits (ONLY IN PRIVATE HISTORY)
There are only two long-term branches that should have active development develop and main. You can see which local branches have been merged/not merged into develop using git branch --merged upstream/develop/git branch --no-merged upstream/develop
- Has all released versions, tagged by version number (following semantic versioning)
- Nothing should be committed to main directly, all changes should be introduced though a
git merge --ff-only - Main only moves forward, no changes to history are allowed.
- Only managers have write permissions.
- Inherited from last commit on main.
- Branch for development. Changes should come through pull requests.
- Should rebase on main whenever main changes (this is why developers should NOT commit directly to develop)
If something went very wrong on the release branch (main) a small branch can be spun out to fix the problem from main, merged into main, and then develop rebased against main. This branch should then be promptly deleted.
These are were the bulk of actual development will occur. This could also be a branch for a particular experiment or developer but that gets more difficult to maintain over time, so as much as possible we should stick to a branch per feature. These branches should be rebased against develop as often as possible to simplify future merging. You're only hurting yourself if you wait to rebase until you are ready to merge.
Rebasing is important to keep new commits together for squashing if deemed necessary.
Whenever the feature is ready it can be merged into develop with a fast forward merge (you've been rebasing regularly right?).
It is recommended to keep feature branches even with upstream/develop after their merge with develop. It will simplify fine tuning in case if the feature represented by the branch will be reverted from the develop for additional development or fixes/corrections.
Feature branches should be deleted as soon as their commits are merged into main via the develop branch.
- Developer: Read/Write access to Feature and Hotfix branches. Read access to main and develop branches
- Manager: Read/Write access to main and develop branches.
-
Setup git configuration:
git config --global branch.autosetuprebase always git config --global user.name "FirsName LastName" git config --global user.email [email protected] git config --global core.ignorecase falseThis will set up the name/email attached to commits you push.
branch.autosetuprebasewill change the default behavior ofgit pullwhen the branch you are on is setup to track another branch. By defaultgit pullwill do agit merge, this changes the behavior so it willgit rebaseinstead. This is useful for feature branches which should always be setup to track upstream/develop. If for some reason you want to use the default behavior of a merge instead of a rebase, you cangit fetchand thengit mergeyour feature branch. Honestly,git pullis two operations masquerading as one, so if you want full control you should probably just avoid it but it works well for keeping a feature branch up to date with minimal fuss. -
Using GitHub fork the main repo (https://github.com/ATTPC/ATTPCROOTv2)
-
Create a local copy of the forked repo:
git clone url_of_forked_repoGitHub no longer lets you authenticate using a password through HTTPS, so you either need to use the HTTP link (https://github.com/UserName/ATTPCROOTv2.git) and setup a Personal Authentication Token or use the SSH link ([email protected]:UserName/ATTPCROOTv2.git) and setup an SSH key to autheticate.
-
Now when your origin points to your fork. You need to add the main repo to your remotes as well. You should have "origin --> you fork" and "upstream --> https://github.com/ATTPC/ATTPCROOTv2.git". You can also use the SSH link for the upstream if desired.
git remote add upstream https://github.com/ATTPC/ATTPCROOTv2.git git fetch upstream -
Create a local develop branch (note this is NOT setup to track upstream/develop).
git checkout -b develop upstream/develop -
Push the local develop branch to your remote (forked repo) and set the tracked remote to origin
git push -u origin develop
-
Create a feature branch from the latest state of the upstream develop.
git fetch upstream git checkout -b featureXXX upstream/develop -
Push the feature branch to your fork and track it
git push -u origin featureXXX
As often as possible sync your feature branch with the central dev.
-
Sync:
git fetch upstream git checkout featureXXX git rebase upstream/develop- Resolve conflicts if any.
- Stage each modified file "git add <file_name>" after conflicts are resolved.
- You can also use "git checkout --theirs/--ours " to help to resolve conflicts.
- Use "git rebase --continue" to continue rebasing.
-
push to you remote clone:
git push origin featureXXXMost probably your local repo and the remote repo will be diverged at this point. Git will warn you that you are about to change the history and will not allow you to push. Please, revise the output to make sure that you are actually pushing to the right repo and only after that execute the following to force git to change the history.
git push -f origin featureXXXWe recommend to do push in two steps intuitionally to prevent unwanted changes. We also recommend you include the branch you are intending to push to reduce the chance of errors. Even if you are 100% sure, ALWAYS first execute
git pushwithout-f. Revise the output. Check that the repo you are pushing is the the one you want and only then force push with-f.
-
Always rebase to the upstream develop branch before requesting to pull.
git fetch upstream git checkout featureXXX git rebase upstream/develop- Resolve conflicts if any.
- Stage each modified file "git add <file_name>" after conflicts are resolved.
- You can also use "git checkout --theirs/--ours " to help to resolve conflicts.
- Use "git rebase --continue" to continue rebasing.
-
Run clang-format on every file you have touched. Check to see if there are any more Doxygen comments you should add.
clang-format -i file1 file2 ...If you lost track you can run clang-format on every cxx/h file in the repository using the script
formatAll.sh$VMCWORKDIR/scripts/formatAll.sh -
Run the static analyzers on the code, and fix any problems introduced. This will be done at the PR stage, and will shorten the time to get the PR approved.
Configure CMake to run the analyzers it has found, and then rebuild the code. From the build folder run:
cmake .. -DRUN_STATIC_ANALYSIS=OFF -DPROJECT_lwyu_DISABLE=ON cmake --build . -j 8 --clean-firstBecause of ROOT's code generation there will be a bunch of false positives in any of the dictionary classes
G__At*.cxx. There is also a bug in IWYU where it likes to suggest the inclusion of a private header (#include <ext/alloc_traits.h> // for __alloc_traits<>::value_type) which should be ignored. Instead add this line to the offending file// IWYU pragma: no_include <ext/alloc_traits.h>If there is an error flagged in your code that is not actually an error (this happens with ROOT and their handling of pointers/memory sometimes) then use comment pragmas to disable the linters. Documentation for those is availible here for iwyu and here for clang-tidy
-
Squash all of your commits. Once your code is perfect, clean up its history.
git rebase -i upstream/developIt is very important for the history of the main repository that all of your commits are squashed. In the future nobody is interested to see your "cosmetic changes" commits or commits related to any other minor changes. The best way to introduce a feature is to introduce it as a patch. This is why you should squash all your commits into one, or few, and write a good proper comment before requesting to pull your code.
-
Push your changes to your remote repo. You may need to use "push -f" since after the rebase your remote repo can be diverged from the local repo.
git push -f origin featureXXX -
Request to pull. Let the managers know that you want your patch to be merged with the central develop branch. The best way is to use GitHub to send a pull request.
Stop working on the featureXXX branch, after you sent a request to pull. Create a new branch for any other feature/ticket/bug.
-
Setup git configuration:
git config --global branch.autosetuprebase always git config --global user.name "FirsName LastName" git config --global user.email [email protected] git config --global core.ignorecase false -
Using GitHub fork the main repo (https://github.com/ATTPC/ATTPCROOTv2)
-
Create a local copy of the forked repo:
git clone url_of_forked_repo -
Now when your origin points to your fork. You need to add the main repo to your remotes as well. You should have "origin --> you fork" and "upstream --> upstream_url". You can also use the SSH link for the upstream if desired.
git remote add upstream upstream_url_ssh_or_https git fetch upstream -
Create a local develop branch (note this is setup to track upstream/develop).
git checkout -t -b develop upstream/develop -
Push the local develop branch to your remote (forked repo) and set the tracked remote to origin
git push -u origin develop
You can process pull requests automatically if GitHub: https://help.github.com/articles/merging-a-pull-request/ ("Rebase and merge")
-
update
git fetch origin git fetch upstream -
Add the developer's repo to your remotes. You need to do it only once per developer, when you for the first time fetch from this developer
git remote add dev_name developerrepo_url git fetch dev_name -
Now merge the changes the developer has provided
git checkout -f develop git rebase upstream/develop git merge --ff-only dev_name/featureXXXIf there are conflicts or git says, that it can't use fast forward, than reject the request and ask developer to rebase from the main dev branchy again, fix conflicts if needed and send a new pull request.
-
If no conflicts are found, push this commit to the main dev branch:
git push upstream develop:develop
The following tip will help us to recover in cases when you or your colleague had to change something in the history of a high level branch (our upstream). It can be easily the case when you need to change history (rebase, move/delete/squash commits, etc.) of the DEV branch, for example. When the history of dev branch is changed, then all branches inherited from it (feature branches) will have problems to rebase on it, because all commits, which differ you from the dev branch will be consider by git as new (your) changes and it will try to merge them.
Let's take an example. Dev before change of the history and F is your feature branch:
~~~~~~~~~~~~~~~~~~~~~
C0
|
C1
|
C2
|\
C3 cf1
C4 cf2
~~~~~~~~~~~~~~~~~~~~~
‘DEV’ has C1,C2,C3,C4 ‘F’ has C1,C2,cf1,cf2
After the change of the history of the DEV it looks like:
~~~~~~~~~~~~~~~~~~~~~
C0
|
C1x
|
C2x
|
C3x
C4x
~~~~~~~~~~~~~~~~~~~~~
And F still looks like:
~~~~~~~~~~~~~~~~~~~~~
C0
|
C1
|
C2
|
cf1
|
cf2
~~~~~~~~~~~~~~~~~~~~~
For git commits C1, C2 is now different from the commits C1x, C2x from the DEV branch. To avoid merging nightmare and duplications of the commits we have to just change the parent commit of our commits in the feature branch. The commits cf1, cf2 is our commits belonging to the new feature. At the moment their parent is C2.
So, what basically happened is that our feature branch is not forked from the changed DEV anymore. But we want it to depend on the DEV. What we need is only to move commits of the feature branch to a new parent - C2x.
Fortunately git can easily help us to fix our problem. Checkout your feature branch you want to fix and execute the following:
~~~~~~~~~~~~~~~~~~~~~
git fetch mainrepo
git rebase --onto mainrepo/dev dev F
~~~~~~~~~~~~~~~~~~~~~
This basically says, “Check out the F branch, figure out the patches from the common ancestor of the mainrepo/dev and our local dev (which is not changed yet) branches, and then replay them onto master.” If you don't have a local, unchanged version of the DEV branch, then you can even manually find out the commit ID, which is the last common between the new DEV and your F branch and execute:
~~~~~~~~~~~~~~~~~~~~~
git rebase --onto <new-parent> <old-parent> F
~~~~~~~~~~~~~~~~~~~~~
This is a speculative guide (tested, but not by someone without me right there to check for mistakes) for moving commits to a feature branch after you've started development on one of the public branches (where stuff should only end up through a pull request). In this example, let's assume there are a number of commits both normal and merge on your fork of the repository. This process will violate the golden rule of collaborative git work, as we will have to rewrite history on the public branch develop has taken place on to return to the same state as the main repository.
Before we do anything, we will create a backup of the current status of the branch we will be pulling commits from and resetting, just in case we break something horribly, and return to the branch we plan on working from.
git checkout develop
git checkout -b develop-backup
git push -u origin develop-backup
git checkout develop
If you have a large number of commits, I would recommend you start by squashing them down into a more manageable number of commits, this would happen anyway before a merge upstream. To squash commits we will use an interactive rebase. To start, make sure you're on the branch you want to move commits from, for this section I'm going to assume the work was don on the branch develop:
git checkout develop
Now we will squash the commits. To start this process we need to find the hash of the oldest commit we want to work on. You can do that using git log --oneline and grabbing the hash, or from GitHub in the commit list. Remember, you can use the shortened version of the hash, you do not need the full SHA512 hash. If you know the number of commits in history you want to re-write/squash you can also use in place of the hash HEAD~5 where the number 5 is replaced with the number of commits you want to go back from your most recent commit on the branch.
git rebase -i hash
This will open a text editor that has a list with each commit on its own line, and each line starts with the word pick. To squash a commit, we need to change the work pick to squash. This will combine the commit with the one immediately above it and add the squashed commit message to the previous commit message. Usually we will also want to rewrite the commit message of the fully squashed commit. For example the text editor might initially look like:
pick f4af10d48 Add missing function implementations. (#123)
pick 3613f45db Update simulated electronic response. (#125)
pick d533d8f24 Fix nullptr exception when map is not set in viewer (#127)
pick e34db6ac3 Add missing functions/enums to linkdefs (#128)
And if we want to squash the last three commits into one, we would do:
pick f4af10d48 Add missing function implementations. (#123)
reword 3613f45db Update simulated electronic response. (#125)
squash d533d8f24 Fix nullptr exception when map is not set in viewer (#127)
squash e34db6ac3 Add missing functions/enums to linkdefs (#128)
Now that we have a cleaner git history will less commits, it is time to move them to a feature branch. To do this, we will use cherry-pick. First, let's get the list of commits we need to move. Using git log --oneline grab and save somewhere the hash of every commit you want to move.
Now, we will need to create a feature branch to apply these commits to. After following the instructions for creating a feature branch, we should have that branch checked out already.
Now, starting with the oldest commit, we will one-by-one cherry-pick the commits to move.
git cherry-pick hash1
git cherry-pick hash2
...
As you cherry-pick each commit, you will have to resolve any merge conflicts from applying these commits on top of the current status of upstream/develop which is what you're feature branch should have spun out of.