Building new package versions - git-for-windows/git GitHub Wiki
As described in Package management, Git for Windows' SDK is a close derivative of the MSYS2 system, and as such, its components are built using makepkg
and makepkg-mingw
(borrowed from Arch Linux' pacman
).
Most of those packages are actually built by the MSYS2 project and consumed by Git for Windows. However, a couple of components (including cURL and OpenSSH) are built by Git for Windows, e.g. to be able to react faster to newly-available versions, or to allow for modifications specific to Git for Windows.
In general, the process to upgrade components to new versions is quite tedious.
First of all, the Git for Windows maintainer needs to be aware that a new version of a Git for Windows SDK component is available. The current process for this involves using a custom IFTTT applet to receive notifications when, say, cURL's Atom feed reports a new version.
The next step is to update the package definitions (i.e. Pacman's PKGBUILD
files), to reflect the new version and the checksum for the respective archive.
Git for Windows offers some automation for that, figuring out the current version of a specified component in its administrative script called please.sh
and updating the respective PKGBUILD
, then building and uploading the Pacman package. The idea is to call e.g. sdk cd build-extra && ./please.sh upgrade curl
to upgrade cURL.
This script is, essentially, the backbone of the two Azure Pipelines used by the Git for Windows project to build new package versions: Build and publish MINGW Pacman package and Build and publish MSYS Pacman package.
However, many a time this automation fails, for a variety of reasons, including, but not limited to:
- Download links do change. If this is the case, the
source
definition in thePKGBUILD
needs to be adjusted. - Patches that are applied as part of the
build
function in thePKGBUILD
script might no longer apply to newer versions (see below). - Spurious network errors. Those are the maintainer's favorite hiccups.
The way makepkg
works is unfortunately not very integrated with Git (as it was invented before Git), therefore the way it works is to specify the URL to the archive (or to the branch in a repository) as source
, together with any patches needed to compile it in MSYS2. The archive will be unpacked in the src/
directory automatically, but the patches need to be applied specifically as part of the prepare
step in the PKGBUILD
.
When these patches no longer apply after upgrading to a newer version of the component, it can be a challenge to fix that.
The strategy that worked best for the Git for Windows maintainer goes like this:
- initialize a new Git repository in
src/playground
- use
/usr/src/git/contrib/fast-import/import-tars.perl
to import the previous version (i.e. the one where the patches still apply) - create a new branch from the import tag (
git switch -c rebase-to-<version> import-tars
) - if the patches are actually in mbox format, they can be imported using
git am ../../*.patch
; otherwise they need to be manually applied and committed (something likesrcdir=../..; grep "patch -p" $srcdir/PKGBUILD | while read line; do patch="${line##*/}" && eval "$line" && git commit -m "$patch" || break; done
) - import the new version using
import-tars.perl
- create a new branch from the current one (
git switch -c rebase-to-<new-version>
) - rebase the patches (
git rebase -i --onto <component>-<new-version> <component>-<version>
) - export the patches (if they were in mbox format,
git format-patch -<N> -o ../..
, otherwise a variation offor commit in $(git rev-list HEAD); do patch="$(git show -s --format=%s $commit)" && git diff $commit^! >../../"$patch" || break; done
) - test (
cd ../.. && sdk build
) - commit, push (and open a PR unless pushing directly to
main
)
Caveat: this strategy needs manual adjustments if the archive in question contains files with DOS line endings (I am looking at you, Perl shakes fist): import-tars.perl
will happily import those verbatim, but the bsdtar
used by makepkg
to unpack the archive will convert them to Unix line endings, and if the patches expect DOS line endings, they won't apply.