Continuous Delivery IDM - noi-techpark/documentation GitHub Wiki

Introduction

To have a common way to handle our git repositories with continuous integration in Jenkins we created this user-guide. Enjoy! :wink:

Configuration for new projects

On Jenkins

Create a new folder for your project with New Item > Folder. Give it a full-qualified name, similar to package names. For instance, a website project for idm-suedtirol.com should be called com.idm-suedtirol.website.

Inside the folder create 3 new items:

  • A multibranch pipeline for continuous integration (name must be <your-project>-CI)
    • Add source > GitHub: credentials = idm-jenkins and owner = idm-suedtirol
    • Buid configuration: mode = by Jenkinsfile and script path = Jenkinsfile-CI (path and file name on repo)
  • A pipeline for archiving or/and deploying on your test-server
  • A pipeline for archiving or/and deploying on your production-server

In general use Pipeline for simple projects, and Multibranch Pipeline for complex ones.

You obviously have the freedom to adapt and optimize your pipelines but please do not forget to document your changes, so that other devs have a chance to understand what you did and why.

On Github

  • Go to Settings > Integrations & services:
    • Add service Jenkins (GitHub Plugin)
    • Jenkins hook url is https://jenkins.testingmachine.eu/github-webhook/
    • Enable it with Active and Update service
  • Go to Branches:
    • Protect branches that should accept commits only if tests pass, mostly just master

Within your repository, create a Jenkinsfile-CI, which will be executed every time a new commit gets added to a branch. It also reports back to GitHub to show if integration succeeded.

Example: Simple maven test script without additional configuration steps

pipeline {
    agent any
    stages {
        stage('Test') {
            steps {
               sh 'mvn test'
            }
        }
    }
}

Nightly Builds on our Test Server

How can I add projects to our nightly builds system?

Go to your project folder in Jenkins and click the + sign above your jobs-list near All. Then add a new List View called Nightly Builds (if not already present). Enable all Jobs inside that view, that you want to have build during night.

NB: Make sure, that you choose build-able jobs, that is no folders or other abstract types. If you want to build jobs of a folder, go there and create a new List View as explained above. Alternatively, you can also choose Recurse in subfolders and select jobs directly from a higher-level folder. Make sure, that you do not choose projects twice this way.

How can I set a dependency for a nightly build, that must be executed before?

Configure your project with This project is parameterized and add a Multi-line String Parameter field

Set Name to DEPENDS and Default Value to <your-dependency> (as full-qualified job name) For instance, it.bz.opendatahub.bigdataplatform/core.testserver-archive

Where can I find the nightly builds execution script?

Go to Jenkins > Server Deployment > Test Server Nightly Builds > Configure > Pipeline

NB: This script uses some Jenkins internals and is therefore not safe to be loaded from Github, hence we must copy/paste the source code from our repository manually and use a simple Pipeline script with disabled Groovy Sandbox.

Configure Jenkins

NB: We need these steps solely for a first-time setup. Ignore this section, if you want to add a new project, or create a new pipeline.

General setup

  • Install plugins from Jenkins web: Git, GitHub, GitHub Branch Source Plugin, and the GitHub Pull Request Builder
  • Global credentials: set secret with id GITHUB_API_TOKEN and the token code as value for a global scope
  • Generate a global jenkins-github user to report back to GitHub and to retrieve source code
    • Credentials > Create new user: ex., id=jenkins-github-access-token, username=_idm-jenkins_ (that is, the github username), password=_github-api-access-token_

NB: Use https://github... address for all projects, so that you can use username + password for authentication.

Email notification

Since it is not possible to retrieve the admin email address easily from within a pipeline script, I took a workaround via credentials. We need to update it every time we choose a new email-address, which should be pretty never.... Little hack, but otherwise we need to search for some more complex solutions or third-party plugins.

Use as:

environment {
        MAIL = credentials('JENKINS_ADMIN_EMAIL')
}

...and then:

mail to: "${MAIL}", 
                subject: "Jenkins: Job '${JOB_NAME}' with build number ${BUILD_NUMBER} is unstable", 
                body: "blabla"

Configure Github

NB: We need these steps solely for a first-time setup. Ignore this section, if you want to add a new project, or create a new pipeline.

  • Generate a GitHub API token under developer settings