ephemeral and permanent POD in Jenkins and Open Shift - unix1998/technical_notes GitHub Wiki

The lifecycle management of Jenkins agent pods created using the Kubernetes plugin is typically controlled by Jenkins itself, rather than directly within the Pod YAML definition. This means that the Pod YAML definition primarily specifies the configuration of the pod and its containers, but not its lifespan.

However, you can influence pod retention behavior using Jenkins pipeline configurations or Kubernetes annotations. Here’s how you can manage the pod's lifecycle and adjust its retention settings:

Adjusting Pod Lifecycle in Jenkins Pipeline

1. Pod Retention Policy via Jenkins Configuration:

   • Jenkins Kubernetes plugin allows you to set a retention policy for the agent pods. By default, the pods are ephemeral and are deleted once the job completes.

2. Pod Template in Jenkinsfile:

   • While defining the pod template in your Jenkins pipeline, you can use the activeDeadlineSeconds setting in the Pod YAML to specify the maximum duration the pod should be active. However, this is more about limiting the pod's run time rather than its lifecycle post-job completion.

3. Jenkinsfile Example with Pod Retention Annotation:

   • You can annotate the pod to retain it for debugging purposes. However, be cautious as this can lead to resource consumption if not managed properly.

Example Jenkinsfile with Pod Lifecycle Management

Here’s how you can incorporate lifecycle management settings within your Jenkins pipeline script:

podTemplate(yaml: '''
apiVersion: v1
kind: Pod
metadata:
  annotations:
    jenkins.io/podRetention: Never # Other options are OnFailure, Always
spec:
  activeDeadlineSeconds: 3600 # 1 hour
  containers:
  - name: maven
    image: maven:3.3.9-jdk-8-alpine
    tty: true
    command:
    - cat
  - name: golang
    image: golang:1.8.0
    tty: true
    command:
    - cat
''') {

    node(POD_LABEL) {
        stage('Get a Maven project') {
            git 'https://github.com/jenkinsci/kubernetes-plugin.git'
            container('maven') {
                stage('Build a Maven project') {
                    sh 'mvn -B clean install'
                }
            }
        }

        stage('Get a Golang project') {
            git url: 'https://github.com/hashicorp/terraform.git'
            container('golang') {
                stage('Build a Go project') {
                    sh """
                    mkdir -p /go/src/github.com/hashicorp
                    ln -s `pwd` /go/src/github.com/hashicorp/terraform
                    cd /go/src/github.com/hashicorp/terraform && make core-dev
                    """
                }
            }
        }
    }
}

Explanation:

• Pod Retention Annotation (jenkins.io/podRetention):

  • Never: The pod is deleted immediately after the job completes.
  • OnFailure: The pod is deleted only if the job succeeds; it is retained if the job fails.
  • Always: The pod is never deleted, useful for debugging but can lead to resource consumption.

• activeDeadlineSeconds:

  • This field limits the active time of the pod to a specified number of seconds. For example, 3600 seconds means the pod will be active for up to 1 hour. This helps in preventing runaway pods that might hang indefinitely.

Adjusting Pod Lifespan Using Kubernetes Annotations:

If you want more control over the pod lifecycle based on Kubernetes settings alone, you can use annotations and other Kubernetes-native configurations. However, for Jenkins-managed pods, it’s best to use Jenkins-specific configurations as shown above.

Summary:

• The pod lifecycle is primarily managed by Jenkins through its pipeline configurations and annotations.
• Adjustments like activeDeadlineSeconds can help manage pod runtime.
• Annotations such as jenkins.io/podRetention in the Pod YAML allow control over pod retention post-job completion.

By configuring these settings, you can effectively manage the lifecycle of Jenkins agent pods, balancing resource utilization and debugging needs.