What is Jenkins?

• Open-source automation server
• Mainly used for Continuous Integration (CI) & Continuous Delivery (CD)
• Helps automate building, testing, and deploying code

Why use Jenkins

• Automation: automate tasks, less efforts, avoid human errors
• Integration: continuously integrate with new code
• Extensibility: many plugins to support different platforms or tools
• Open-source

Jenkins Architecture

Jenkins master

• Serve as Jenkins admin UI
• Distribute build jobs to agents
• Monitor agents

Agent:

• Machine that execute the job: build, test, deploy,...
• Report results back to master

Executor:

• A "computational resource" for running a build
• Example: if an agent has 2 executors, it can run 2 build jobs simultaneously

Setting up Jenkins

Naive approach

This approach aims to setup Jenkins to work quickly

• Master & Agent: use a server to run the Jenkins master. E.g: we run the Jenkins JAR on EC2 machine
• Executor: use the above server also as executors. Build jobs will be executed right on this server
• Tools: to build, test and deploy, the tools are installed directly on server. Operators with access will remote control to the server and manually install. E.g: Java, Maven, Docker, kubectl, AWS CLI...

Pros

• Quick and simple to setup
• Minimize resource usage
• Build speed can be improved due to pre-installed tools and cached data after builds ( e.g: maven cache, docker cache)

Cons

• Difficult to scale
• Single Point of Failure

A K8S based setup

This setup leverage K8S environment to setup Jenkins pipeline smoothly

• Master: deploy to K8S using container or using Helm Chart
• Agent: using kubernetes agent in Jenkinsfile. kubernetes agent will specify the container(s) of the pod, and create the deployment for Agent during build execution
• Tools: The agent will need to install the tools manually, either by pre-installed in the image or using the tools declaration

Pros

• Easy to scale
• Configurations could be reused

Cons

• Very complex to beginners

Pipelines

• Structure your deployment into stages
• Each stages contains many steps
• Builds can be retried from a specific stage

Types of pipelines

• Pipelines are often defined in Jenkinsfile
• This file could be defined directly on the Jenkins Pipeline, but normally stored in SCM

Declarative pipelines

• Using Pipeline DSL syntax

pipeline {
    agent any  // This means the build can run on any available agent

    tools {
        // Define tools, for example:
        // maven 'Maven 3.6.0'
    }

    environment {
        // Define environment variables, for example:
        // MY_ENV_VAR = 'Some Value'
    }

    stages {
        stage('Build') {
            steps {
                // Define build steps, for example:
                // sh 'make'
            }
        }

        stage('Test') {
            steps {
                // Define test steps, for example:
                // sh 'make test'
            }
            post {
                always {
                    // Actions to perform after this stage, regardless of its completion status
                    // For example, archiving test results
                }
                success {
                    // Actions to perform only when the stage completes successfully
                }
                failure {
                    // Actions to perform only when the stage fails
                }
            }
        }

        // You can define more stages as needed
    }

    post {
        always {
            // Actions to perform after all stages, regardless of their completion status
        }
        success {
            // Actions to perform only when all stages complete successfully
        }
        failure {
            // Actions to perform only when any stage fails
        }
    }
}

Scripted pipelines

• Use Groovy scripting
• Starts with a node block

node {
    def myVar

    stage('Checkout') {
        checkout scm
    }

    stage('Build') {
        myVar = 'Some value'
        sh 'make'
    }

    stage('Test') {
        if (someCondition) {
            sh 'make test'
        } else {
            echo 'Skipping tests'
        }
    }
}

Builds

• Manual Build
• Parameterized Build: we can define some input parameters, and they can be used as environment variables in pipeline script
• Remote Build: cal JENKINS_URL/job/PIPELINE_NAME/build?token=TOKEN_NAME or /buildWithParameters?token=TOKEN_NAME

Integrations

SCM

Pull source code

• To pull source code from a private repository, an SCM credentials should be setup in Jenkins web UI
• Use the specified credentials to clone repository

Build triggers

• For most SCM, there will be triggers for source code related events
• Setup the SCM webhooks with the URL of Jenkins SCM hook

Sonar

Push code inspection data to Sonar

• A Sonar token should be registered as secret

• Inject the Sonar token during the build and test process

withSonarQubeEnv('dev') {
  sh "mvn clean verify sonar:sonar -Pcoverage -Dsonar.projectKey=${SONAR_PROJECT_KEY} -Dsonar.projectName='$SONAR_PROJECT_NAME'"
}

Get quality result from Sonar

• Provide Sonar with webhook URL of Jenkins (provided by Sonar plugin)
• Define a step to wait for Quality Gates

stage('Quality Gate Check') {
      steps {
          timeout(time: 10, unit: 'MINUTES') {
              // Just in case something goes wrong, pipeline will be killed after a timeout
              script {
                def qg = waitForQualityGate() // Reuse taskId previously collected by withSonarQubeEnv
                if (qg.status != 'OK') {
                  error "Pipeline aborted due to quality gate failure: ${qg.status}"
                }
              }
          }
      }
    }

Notifications

Send build results via Slack

• Authenticate Jenkins to Slack to allow Jenkins posting data
• Add notifications to post section so the notifications are always sent

post {
    failure {
      slackSend message: "Build Failure: ${env.JOB_NAME} ${env.BUILD_NUMBER} (${env.BUILD_URL})", color: 'danger'
    }
    success {
      slackSend message: "Build Success: ${env.JOB_NAME} ${env.BUILD_NUMBER} (${env.BUILD_URL})", color: 'good'
    }
  }

A Sample Setup