Pekko Release Process - apache/pekko-site GitHub Wiki

The release is managed by a Release Manager (a human) and involves building and deploying artifacts (including generating SHA digests and PGP signatures for them). Emails must be sent to Apache mailing lists and a vote arranged.

The release can only be finalized after passing the vote. This involves moving the release artifacts to official Apache release repo (from the staging repo). We also need to release the Maven artifacts. The Pekko web site must be updated to include links to download the release. Emails are also sent out announcing the release.

If the component you're releasing has already been set up for CI-assisted releasing, see Pekko-Release-Process ‐ ATR-CI‐assisted-release instead.

Relevant Reading

Old Incubator links

Agree to start the Release Process

  • Pekko dev mailing list

Checklist

  • You can start the release notes PR early, but we no longer require it to be merged before tagging the release.
  • Releasing off a branch other than main branch?
    • We don't tend to backport changes like year changes in NOTICE files and dependency version updates
    • so if you are releasing from a branch like 1.0.x, 1.1.x, etc., you should check if we need to update the year in the NOTICE files (check also the 'legal' directory)
    • see if there are patch level dependency updates that can be added - maybe only if they fix CVEs and only if they don't require much uptake other than changing changing the jar version

Build Prerequisites

  • There might be a Docker image created but please don't remove the instructions not related to Docker
    • it is useful for people to understand how to do the build even if it is partially encapsulated in a Docker image
  • Install the Java SDK (which versions are acceptable may depend on which module you're releasing)
  • Install sbt
  • Install GraphViz - version 2.26 or newer required

GPG

sbt setup

  • Add credentials += Credentials(Path.userHome / ".sbt" / "sonatype_credentials") to your $HOME/.sbt/1.0/sonatype.sbt

  • Add your Apache credentials to $HOME/.sbt/sonatype_credentials (if the file already exists then append it after existing entries).

realm=Apache Nexus Repository Manager
host=repository.apache.org
user=<APACHE ID>
password=<APACHE PASSWORD>

Build the Source Release (Candidate)

  • Use git status to ensure you are using the 'main' branch and have no uncommitted file changes
    • git tag - eg v1.0.0-RC1
  • sbt "clean; set ThisBuild / version := \"1.0.0\"; signedSourceDistGenerate" (or sbt "clean; set ThisBuild / version := \"1.0.0\"; sourceDistGenerate")
    • NB do not include the 'RC' in version name when building the source distribution
    • these files need to be released without modification after the vote passes
  • The generated files will appear in target/dist directory
  • If you used sourceDistGenerate above, use gpg -ab to generate .asc signatures. You may need to use the --default-key <SECRET_KEY_ID> option.
  • Add the files to svn - https://dist.apache.org/repos/dist/dev/pekko (create a dir named after the RC version, eg 1.0.0-RC1).
    • svn co https://dist.apache.org/repos/dist/dev/pekko/ pekko-dist
    • cd pekko-dist
    • mkdir 1.0.0-RC1
    • cd 1.0.0-RC1
    • cp /path/to/apache-pekko-1.0.0-src-20230527.tgz .
    • cp /path/to/apache-pekko-1.0.0-src-20230527.tgz.asc .
    • cp /path/to/apache-pekko-1.0.0-src-20230527.tgz.sha512 .
    • cd ..
    • svn commit -m "Pekko 1.0.0-RC1"
  • If you leave the steps above to CI, still build the archive locally and compare the result.

Deploy the jars to Apache Maven Repository (staging)

  • we are not yet releasing, we are just staging the jars for later release
  • use git status to ensure you are using the 'main' branch and have no uncommitted file changes
  • Use the appropriate JDK when running these sbt commands
    • double check by running java -version and watching the first line of the sbt startup
  • Make sure that the project is clean by running sbt +clean.
    • This is important as sbt's incremental compiler is not deterministic if existing compiled JVM bytecode exists
  • sbt "set ThisBuild / version := \"1.0.0-RC1\"; +publishSigned"
  • before proceeding to the Sonatype upload, please check the output from the publishSigned run
    • there might be plenty of warnings about scaladoc but check to see if there were issues with graphviz calls
    • if you have graphviz errors, you should check that you have an up to date version installed
    • if you continue to have graphviz issues, please contact the Pekko dev list before proceeding
  • be very careful with the sbt-sonatype calls
    • we cannot do a full release until the votes pass
    • sonatypeRelease and sonatypeBundleRelease are the commands that MUST NOT be used
  • sbt "set ThisBuild / version := \"1.0.0\"; sonatypePrepare; set ThisBuild / version := \"1.0.0\"; sonatypeBundleUpload; sonatypeClose"
    • sometimes the 'close' fails due to sonatype not finding your public key in the keyservers, even when they are there. In that case, log into https://repository.apache.org with your Apache credentials, go to 'Staging Repositories', find the relevant repository, and manually trigger the 'close' again.
  • If you used CI to achieve the above, verify the binaries can be reproduced

Organise Pekko review and vote

Subject: [VOTE] Release Apache Pekko 1.0.0-RC1

Hello Pekko Community,

This is a call for a vote to release Apache Pekko version 1.0.0-RC1.

The discussion thread:

<insert lists.apache.org link here>

The release candidate:

https://dist.apache.org/repos/dist/dev/pekko/1.0.0-RC1/

This release has been signed with a PGP key available here:

https://downloads.apache.org/pekko/KEYS

Release Notes:

<insert URL here>

Git branch for the release:

https://github.com/apache/pekko/tree/v1.0.0-RC1
Git commit ID: <insert ID here> 

Please download, verify, and test.

We have also staged jars in the Apache Nexus Repository. These were built with the same code
as appears in this Source Release Candidate. We would appreciate if users could test with these too.
If anyone finds any serious problems with these jars, please also notify us on this thread.

https://repository.apache.org/content/groups/staging/org/apache/pekko/

For sbt 1.9.4 or greater you can add this resolver

resolvers += Resolver.ApacheMavenStagingRepo

Otherwise for older versions of sbt

resolvers += "Apache Pekko Staging" at "https://repository.apache.org/content/groups/staging"


The VOTE will pass if we have more positive votes than negative votes
and there must be a minimum of 3 approvals from Pekko PMC members.
Anyone voting in favour of the release, could you please provide a list of the checks you have done?
The vote will be left open until <insert date/time here>.

[ ] +1 approve
[ ] +0 no opinion
[ ] -1 disapprove with the reason

To learn more about Apache Pekko, please see https://pekko.apache.org/

Checklist for reference:

[ ] Download links are valid.
[ ] Checksums and signatures.
[ ] LICENSE/NOTICE files exist
[ ] No unexpected binary files
[ ] All source files have ASF headers
[ ] Can compile from source
[ ] Can verify the binary build

To compile from the source, please refer to:

https://github.com/apache/pekko/blob/main/README.md#building-from-source

To verify the binary build, please refer to:

https://github.com/apache/pekko-site/wiki/Pekko-Release-Process#verifying-the-binary-build

Some notes about verifying downloads can be found at:

https://pekko.apache.org/download.html#verifying-downloads


Here is my +1.

Thanks,

XXXXXXX (Apache Pekko PMC member)
  • include the release notes
  • details about the Maven staged jars and how to construct a resolver for sbt (maybe also mvn)
  • basic details about the voting process and the date/time of when the vote ends

Pekko Vote Rules

  • anyone can vote
  • we need a minimum of 3 approvals from Pekko PMC members
  • if there were artifacts staged from CI, the review must include independently reproducing these artifacts.
  • if we have more approvals than disapprovals, the vote passes (as long we have more approvals than disapprovals among all voters), eg 3 +1s and 2 -1s is a vote for release
  • the vote can be cancelled by the Release Manager if there is a serious flaw found in the release candidate

If Pekko Vote Fails

Build the Release

  • add git tag 'v1.0.0' to the same commit as the accepted RC and push it

  • ensure that this tag is what you have checked out and that you have no uncommitted changes locally

  • Deploy the jars to Apache Maven Repository (staging) building with version 'v1.0.0'

    • leave the 1.0.0-RC1 jars in staging for comparison
    • the steps are the same as when we deployed the RC (see above)
    • again, do not complete the Maven release yet

  • email the Pekko dev thread to say that release jars are staged and invite users to double check that they match the RC jars.

  • Leave the 1.0.0 jars up at least 24 hours so that the community can verify that they match the latest successfully voted RC jars

    • we require at least 1 PMC member to validate the release jars match the RC jars

Verifying the binary build

  • Pekko projects use sbt-reproducible-builds as a tool to verify that the jars built in the Build the Release step can be reproduced and hence verified by the community.
    • The tool works by creating binaries locally and comparing them against a Maven repository
    • you may need to install diffoscope which sbt-reproducible-builds uses to diff the files
    • By default the Pekko projects check against Apache Staging repo (i.e. https://repository.apache.org/content/groups/staging/) since this is where binaries are meant to sit while the voting is happening
      • If needed its possible to override this, i.e. by running sbt "set ThisBuild / reproducibleBuildsCheckResolver := Resolver.DefaultMavenRepository; shell"
        • Note that its important that you override the ThisBuild / reproducibleBuildsCheckResolver in this manner because otherwise the project will be in a dirty VCS state which causes the project to build as a snapshot
    • First run sbt +clean and then run sbt ++2.12 reproducibleBuildsCheck/sbt ++2.13 reproducibleBuildsCheck respectively to check that your binary build matches what the release manager has deployed.
    • Then check sbt's output to see if there are any potential problems, i.e. 
      [info] # `pekko-bench-jmh_2.12-1.0.1-RC1.buildinfo`: NOT OK
[info]
[info] - pekko-bench-jmh_2.12-1.0.1-RC1.jar: Missing in theirs but present in ours
[info] - pekko-bench-jmh_2.12-1.0.1-RC1.pom: Missing in theirs but present in ours
      Is usually a red herring since the tool will try and check against all possible artifacts (even though some are intentionally not designed to be deployed), see https://github.com/raboof/sbt-reproducible-builds/issues/257. What you need to look out for are cases like this 
      [info] - pekko-remote_2.13-1.0.1.jar: Mismatch: our 95c34c631ecc11ffd02b9d823977167e72dbe5ebd9127d15e68c7bd0173c79f144f5bd8391f74e1b641f973a62fb3755c7ce0669e04394b5b44d6bca63dc424a did not match their 8b3bbdf2c962b23f49d493da1bd3f43de7bfe0d7b19399133fd549dc9fb7d3c464258e44d8d15d489f7ca44547cabe35698fde270816f537111e78a2817a8820
      Then it indicates there was a problem.

Complete the Release

⚠️ **GitHub.com Fallback** ⚠️