git workflows - HeritageNetwork/Regional_SDM GitHub Wiki

All of the discussion here applies to the main, aquatic, and terrestrial branches. terrestrial is used in the examples, but the other branches can be applied interchangeably.

Table of Contents

Get copy of repository
Helpful git commands
A simplified PR workflow
A PR workflow that maintains local edits
Ways to update without losing your work

Get the repository

expand

To create a new instance of the repository so you can start modeling:

1. fork the repository

  • If you don't have one already, create a github account (here: https://github.com), log in.

    • examples here will show username: empetrum
  • Navigate back to https://github.com/HeritageNetwork/Regional_SDM, choose Fork (button near upper-right). A fork is created in your user space and you'll see something like this in the upper left of the page when you are on the fork:

    image showing repo fork

  • Here, at your copy of the repo on github, click the Code button.

  • Choose HTTPS click the clipboard to copy the path.

2. clone your fork on the VM

  • On the computer (or virtual machine) where you'll be doing the modeling, navigate, using Windows Explorer, to the location where you want the new copy of the repository
  • Right-click inside the right panel of Windows Explorer, choose Git Bash Here. If this option isn't available, install git, see https://github.com/HeritageNetwork/Regional_SDM/wiki/Software
  • now use the HTTPS path you saved (above) to get a copy (clone) of your repository, preface it with git clone like this:
git clone https://github.com/empetrum/Regional_SDM.git
  • it should look like this:

git clone command and results

  • Now you've got a full copy of the repo, in a folder named 'Regional_SDM' on your workstation.
  • Type cd Regional_SDM to "change directory" into the repository folder.

3. Add some git settings

  • add the HeritageNetwork version of the repo as another "remote", call it upstream.
  • git remote add upstream https://github.com/HeritageNetwork/Regional_SDM
Tim_Howard@MoBI01 MINGW64 /g/Users/tim_howard/Regional_SDM (main)
$ git remote add upstream https://github.com/HeritageNetwork/Regional_SDM

Helpful git commands

expand
  • git checkout <branch> aligns your files to a specific branch.
Tim_Howard@MoBI01 MINGW64 /g/Users/tim_howard/Regional_SDM (main)
$ git checkout terrestrial
Switched to a new branch 'terrestrial'
Branch 'terrestrial' set up to track remote branch 'terrestrial' from 'origin'.
  • git status tells you where you are and what's going on
Tim_Howard@MoBI01 MINGW64 /g/Users/tim_howard/Regional_SDM (terrestrial)
$ git status
On branch terrestrial
Your branch is up to date with 'origin/terrestrial'.

nothing to commit, working tree clean
  • git branch tells you what branches are in your repo. The star "*" shows what branch is checked out.
Tim_Howard@MoBI01 MINGW64 /g/Users/tim_howard/Regional_SDM (terrestrial)
$ git branch
  main
* terrestrial
  • git remote -v tells you what remotes you have set up. If you don't see the upstream branch, add it as described in #3 in the section above
Tim_Howard@MoBI01 MINGW64 /g/Users/tim_howard/Regional_SDM (terrestrial)
$ git remote -v
origin  https://github.com/empetrum/Regional_SDM.git (fetch)
origin  https://github.com/empetrum/Regional_SDM.git (push)
upstream        https://github.com/HeritageNetwork/Regional_SDM (fetch)
upstream        https://github.com/HeritageNetwork/Regional_SDM (push)

  • git log shows you the last few commits of the branch you are working on. It also gives you another prompt : which is confusing. At this prompt, if you hit the space bar, the log will advance a page; Enter will advance one line. To quit this prompt, type q. This bug may have been fixed, but if you accidentally type something else (such as ctrl-c) and return to the git prompt, but then you can't see your typing, type reset press enter and you should be back in order (https://stackoverflow.com/a/43267044/435082 and https://stackoverflow.com/questions/44281617/git-text-invisible).

Simplified PR workflow

expand

Overview: make a change, push it to your repo, PR it to HeritageNetwork repo

If you make a change you want to be applied to the main repository, follow these steps:

  1. make the edit
  2. commit the change
  3. push that commit up to your own repo
  4. submit a pull request (PR) on the main repo. An owner 'accepts' the changes and completes the pull
  5. you then pull the updated version of the repo back to your local repository.

Here's a diagram of that workflow

simplified workflow

Here are the commands.

  1. Make your edit. Here, I'll add a package to the pkg_check.R script.
  2. Commit your change. This is easier in RStudio, but showing here using command line.
    • The command is git commit -m "update required packages". All commits require a message; I've added it here in-line for simplicity.
    • You first add the change git add helper/pkg_check.R, then commit it.
      (git has autocomplete, so you can actually type git add h<tab>p<tab>)
      git commit example
  3. Using command line, push your change up to your repo on github\
    • the command is git push origin terrestrial "origin" is your remote repo, "terrestrial" is the branch you are pushing. Warning: use the full specification, not the shorthand "git push" because you have multiple remotes and you don't know where the unattributed push will go!\
    • when asked for a password, enter your Personal Access Token. It should save it and you should be good from now on.
      git push
  4. Now go up to your repo on github, you'll likely see a message like this:
    recent push notification
    • click Compare & pull request, then make sure the repos and branches are lined up like this, with the HeritageNetwork repo on the left and your repo on the right, and both of them are noting the correct branch. Add a title and comment on your changes.
      create pull request
    • After filling this out, click the green button: pull request button
    • I (and other owners, I think) get a message, we check it out, "approve" it, perhaps have a dialogue, then complete the pull request.
  5. Because the pull is a commit, your local repo is out of sync with the HeritageNetwork repo, so, finally, pull down the current state of the repo to sync up. The command is git pull upstream terrestrial
    git pull upstream
  6. Full loop complete!

This is how github diagrams the same workflow (at the repo on github, choose Insights -> Network). The straight yellow line is the terrestrial branch. Another user (Patrick) pulled the current version to his branch (arrow up to blue line), he then made some edits and submitted them back to the terrestrial branch, where they were merged back in (blue arrow down to yellow line).
simplifield workflow schema

The problem with a direct application of this workflow to our needs is that we also make changes to the code that we do not want pushed up to the HeritageNetwork repository. That adds extra complexity and requires us to step up to a whole new level.

PR workflow to maintain local edits

expand

How to maintain our custom changes we don't want to be pushed up to the Heritage Network repository?

Overview: maintain a custom branch, make a change, cherry pick that change to your local copy of terrestrial, push it to your repo, PR it to HeritageNetwork repo, as follows:

  1. make a change in 'myProjectRepo'
  2. commit the change
  3. cherry-pick that commit to your local terrestrial branch
  4. push that commit up to your own repo
  5. submit a pull request (PR) on the main repo. An owner 'accepts' the changes and completes the pull
  6. you then pull the updated version of the repo back to your local repository.

Diagramming it: complex workflow (powerpoint)

Commands in detail:

  1. For completeness, I'll first make a new branch to keep my project-only modifications in, named myProj.
    To do that, use git checkout -b myProj.
    See here for more https://git-scm.com/book/en/v2/Git-Branching-Basic-Branching-and-Merging.
    git checkout new branch
  2. This branch has changes (to script 0, for example), that you don't want sent up to the main repo. Commit your changes separately so that the script 0 changes are in one commit (or more) and the change you want is in another commit. Here's an example for doing just that:
    • git status to see that two files are changed
    • git add and git commit to commit the project-specific tweaks
    • git add and git commit to commit the fix we want pushed up to the main repo
      git commit proj and package changes
  3. You need the commit ID for the one you want to push up to the main repo.
    * Use git log for that, and then copy the commit ID (copy-paste is funny in command-line windows)
    * q at the : to quit the git log prompt
    * then switch to the terrestrial branch git checkout terrestrial
    * git cherry-pick to get pull in the commit.
    git log
    git cherry-pick from myProj
  4. Now follow the same workflow as above, git push to your repo online. Again, use the full specification: git push origin terrestrial.
  5. Submit a PR to the main repo
  6. Pull the newly updated repository down to your terrestrial branch
  7. I'll cover getting changes back into your project branch in the next section.

Update from main repository without losing your work

expand

There are many options, here are a few:

  1. git cherry-pick the commits
  2. git merge the changes.
  3. git stash then git merge then git stash pop
  4. Merge your custom changes into a new copy of the terrestrial branch. Rename your project branch, make a new branch from terrestrial, apply your custom changes onto this new branch.

In more detail:

  1. git cherry-pick the commit.
    * if there are only a few commits you want to get, this might be simplest.
    * from the terrestrial branch, use git log to see the commit ID values. Then q to quit from the log prompt.
    * The commit IDs are now on screen so you can switch to your branch and copy-paste in each commit after a git cherry-pick\
  2. git merge the changes.
    * Any file where there are changes on both branches will ask require you to edit those files and accept the changes from the branch you want.
    * Simply edit the file, accepting the text from your branch. More details here: https://git-scm.com/docs/git-merge#_how_conflicts_are_presented.
    * After fixing those conflicts, finish the merge. Example commands:
    git merge into my proj
  3. git stash then git merge then git stash pop
    * Another way to deal with this is to not commit the changes you want to keep local. This can get confusing and possibly more difficult to track, but git has a way of "stashing" your changes that you don't want to commit and then giving you the opportunity to get them back later.
    * More here: https://github.com/tghoward/images_test/blob/main/git_merge_into_myProj.png. Probably the most straightforward way to do this is:
    * git stash. Saves all your current changes in a special place. Allows you to leave the branch and do other things.
    * git merge terrestrial. Bring in the current state of the terrestrial branch.
    * git stash pop. Apply the stashed changes back on to your branch. Also remove these changes from the stash "place". This is a merge, so if there are conflicts, it will let you know and mark them.
    git stash example
  4. Merge your custom changes into a new copy of the terrestrial branch.
    * The idea here is that you want to make absolutely sure all your changes are saved and applied on top of the scripts in the main repo.
    * rename your current custom branch so you can keep using the same name git branch -m myProj myProjOld
    * make a new branch of the most recent version of terrestrial git checkout -b myProj (when in terrestrial)
    * merge in myProjOld git merge myProjOld
    * delete myProjOld git -d myProjOld
    git move and merge

Here's a schematic of this last method. This method might make the most sense when a big update needs to be applied.

git merge into new base

Finally, remember that googling git commands can be very helpful - I usually find the stackOverflow sites the most helpful. Also the Pro-Git book is fully online and it and the git documentation are also excellent. https://git-scm.com/book/en/v2 .

⚠️ **GitHub.com Fallback** ⚠️