Launch Jenkins on Kubernetes Use "PodTemplate" as a WorkAround - q-uest/notes-doc-k8s-docker-jenkins-all-else GitHub Wiki

-- Issues/bugs with the default "jnlp" container used by Kubernetes plugin to launch an agent in Jenkins

Errors/Bugs logged in the Jenkins master pod:

io.fabric8.kubernetes.client.KubernetesClientTimeoutException: Timed out waiting for [100000] milliseconds for [Pod] with name:[default-m1q4w] in namespace [monitoring].
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilCondition(BaseOperation.java:860)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilReady(BaseOperation.java:842)
        at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilReady(BaseOperation.java:83)
        at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:173)
        at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:298)
        at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:48)
        at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:82)
        at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at java.base/java.lang.Thread.run(Thread.java:829)
2022-12-09 08:06:44.616+0000 [id=48880] INFO    o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance for agent default-m1q4w
2022-12-09 08:06:44.655+0000 [id=48880] INFO    o.c.j.p.k.KubernetesSlave#deleteSlavePod: Terminated Kubernetes instance for agent monitoring/default-m1q4w
2022-12-09 08:06:44.655+0000 [id=48880] INFO    o.c.j.p.k.KubernetesSlave#_terminate: Disconnected computer default-m1q4w
2022-12-09 08:06:44.658+0000 [id=9138]  INFO    o.c.j.p.k.p.r.Reaper$RemoveAgentOnPodDeleted#onEvent: monitoring/default-m1q4w was just deleted, so removing corresponding Jenkins agent

Objervation: As long as not making any changes to the default Resource Requests/Limits to the container template [Available at "Manage Jenkins" -> "Manage Node & Clouds" -> "Configure Clouds" -> "Pod Templates" -> "Pod Template Details" -> "Advanced" ], it works well. If you make changes to it and then execute any job, the slave/Agent pod is not getting created and it goes hanging forever. The reason for this is yet to be found.

Work Around: Use "podTemplate" ; Generate it with required parameters from "Pipeline Syntax":

podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '', command: '', image: 'jenkins/inbound-agent:latest', livenessProbe: containerLivenessProbe(execArgs: '', failureThreshold: 0, initialDelaySeconds: 0, periodSeconds: 0, successThreshold: 0, timeoutSeconds: 0), name: 'jnlp',resourceLimitCpu: '500m', resourceLimitEphemeralStorage: '', resourceLimitMemory: '1000Mi', resourceRequestCpu: '500m', resourceRequestEphemeralStorage: '', resourceRequestMemory: '1000Mi', workingDir: '/home/jenkins/agent')], namespace: 'jenkins') {

The below simple pipeline job works fine when added to the above podTemplate:

podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '', command: '', image: 'jenkins/inbound-agent:latest', livenessProbe: containerLivenessProbe(execArgs: '', failureThreshold: 0, initialDelaySeconds: 0, periodSeconds: 0, successThreshold: 0, timeoutSeconds: 0), name: 'jnlp',resourceLimitCpu: '500m', resourceLimitEphemeralStorage: '', resourceLimitMemory: '1000Mi', resourceRequestCpu: '500m', resourceRequestEphemeralStorage: '', resourceRequestMemory: '1000Mi', workingDir: '/home/jenkins/agent')], namespace: 'jenkins') 
{
 node(POD_LABEL) {
 stage('Get a Maven project') 
 {
   container('jnlp') 
  {
    stage('Shell Execution') 
    {
      sh '''
      echo "Hello! I am executing shell"
      '''
    }
  }
  }
}
}

Example 2 : Pipeline job using SonarQube & gradle

Reference: https://github.com/q-uest/Notes-Jenkins/wiki/Jenkins-Pipeline-App-Job-with-Sonarqube-on-K8s-cluster SCM Repo URL: https://github.com/q-uest/CICD_Java_gradle_application.git

podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '', command: '', image: 'jenkins/inbound-agent:latest', livenessProbe: containerLivenessProbe(execArgs: '', failureThreshold: 0, initialDelaySeconds: 0, periodSeconds: 0, successThreshold: 0, timeoutSeconds: 0), name: 'jnlp', resourceLimitCpu: '500m', resourceLimitEphemeralStorage: '', resourceLimitMemory: '1000Mi', resourceRequestCpu: '500m', resourceRequestEphemeralStorage: '', resourceRequestMemory: '1000Mi', workingDir: '/home/jenkins/agent')], namespace: 'jenkins') 
{
    node(POD_LABEL) {
     stage("Pipeline job starts")
     {
        environment{
        VERSION = "${env.BUILD_ID}"
       }
       stage("checkout scm") {
        git branch: 'main', url: 'https://github.com/q-uest/CICD_Java_gradle_application'
        }
       stage("sonar uality check"){
            script{
                    withSonarQubeEnv(credentialsId: 'sonarqube-id') {
                            sh 'chmod +x ./gradlew'
                            sh './gradlew sonarqube'     
               }
                      def qg = waitForQualityGate()
                      if (qg.status != 'OK') {
                           error "Pipeline aborted due to quality gate failure: ${qg.status}"
                      }
                }  
            }
        
       }
    }
}