Launch Jenkins on Kubernetes Use "PodTemplate" as a WorkAround - q-uest/notes-doc-k8s-docker-jenkins-all-else GitHub Wiki
-- Issues/bugs with the default "jnlp" container used by Kubernetes plugin to launch an agent in Jenkins
Errors/Bugs logged in the Jenkins master pod:
io.fabric8.kubernetes.client.KubernetesClientTimeoutException: Timed out waiting for [100000] milliseconds for [Pod] with name:[default-m1q4w] in namespace [monitoring].
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilCondition(BaseOperation.java:860)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilReady(BaseOperation.java:842)
at io.fabric8.kubernetes.client.dsl.base.BaseOperation.waitUntilReady(BaseOperation.java:83)
at org.csanchez.jenkins.plugins.kubernetes.KubernetesLauncher.launch(KubernetesLauncher.java:173)
at hudson.slaves.SlaveComputer.lambda$_connect$0(SlaveComputer.java:298)
at jenkins.util.ContextResettingExecutorService$2.call(ContextResettingExecutorService.java:48)
at jenkins.security.ImpersonatingExecutorService$2.call(ImpersonatingExecutorService.java:82)
at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:829)
2022-12-09 08:06:44.616+0000 [id=48880] INFO o.c.j.p.k.KubernetesSlave#_terminate: Terminating Kubernetes instance for agent default-m1q4w
2022-12-09 08:06:44.655+0000 [id=48880] INFO o.c.j.p.k.KubernetesSlave#deleteSlavePod: Terminated Kubernetes instance for agent monitoring/default-m1q4w
2022-12-09 08:06:44.655+0000 [id=48880] INFO o.c.j.p.k.KubernetesSlave#_terminate: Disconnected computer default-m1q4w
2022-12-09 08:06:44.658+0000 [id=9138] INFO o.c.j.p.k.p.r.Reaper$RemoveAgentOnPodDeleted#onEvent: monitoring/default-m1q4w was just deleted, so removing corresponding Jenkins agent
Objervation: As long as not making any changes to the default Resource Requests/Limits to the container template [Available at "Manage Jenkins" -> "Manage Node & Clouds" -> "Configure Clouds" -> "Pod Templates" -> "Pod Template Details" -> "Advanced" ], it works well. If you make changes to it and then execute any job, the slave/Agent pod is not getting created and it goes hanging forever. The reason for this is yet to be found.
Work Around: Use "podTemplate" ; Generate it with required parameters from "Pipeline Syntax":
podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '', command: '', image: 'jenkins/inbound-agent:latest', livenessProbe: containerLivenessProbe(execArgs: '', failureThreshold: 0, initialDelaySeconds: 0, periodSeconds: 0, successThreshold: 0, timeoutSeconds: 0), name: 'jnlp',resourceLimitCpu: '500m', resourceLimitEphemeralStorage: '', resourceLimitMemory: '1000Mi', resourceRequestCpu: '500m', resourceRequestEphemeralStorage: '', resourceRequestMemory: '1000Mi', workingDir: '/home/jenkins/agent')], namespace: 'jenkins') {
The below simple pipeline job works fine when added to the above podTemplate:
podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '', command: '', image: 'jenkins/inbound-agent:latest', livenessProbe: containerLivenessProbe(execArgs: '', failureThreshold: 0, initialDelaySeconds: 0, periodSeconds: 0, successThreshold: 0, timeoutSeconds: 0), name: 'jnlp',resourceLimitCpu: '500m', resourceLimitEphemeralStorage: '', resourceLimitMemory: '1000Mi', resourceRequestCpu: '500m', resourceRequestEphemeralStorage: '', resourceRequestMemory: '1000Mi', workingDir: '/home/jenkins/agent')], namespace: 'jenkins')
{
node(POD_LABEL) {
stage('Get a Maven project')
{
container('jnlp')
{
stage('Shell Execution')
{
sh '''
echo "Hello! I am executing shell"
'''
}
}
}
}
}
Example 2 : Pipeline job using SonarQube & gradle
Reference: https://github.com/q-uest/Notes-Jenkins/wiki/Jenkins-Pipeline-App-Job-with-Sonarqube-on-K8s-cluster SCM Repo URL: https://github.com/q-uest/CICD_Java_gradle_application.git
podTemplate(cloud: 'kubernetes', containers: [containerTemplate(args: '', command: '', image: 'jenkins/inbound-agent:latest', livenessProbe: containerLivenessProbe(execArgs: '', failureThreshold: 0, initialDelaySeconds: 0, periodSeconds: 0, successThreshold: 0, timeoutSeconds: 0), name: 'jnlp', resourceLimitCpu: '500m', resourceLimitEphemeralStorage: '', resourceLimitMemory: '1000Mi', resourceRequestCpu: '500m', resourceRequestEphemeralStorage: '', resourceRequestMemory: '1000Mi', workingDir: '/home/jenkins/agent')], namespace: 'jenkins')
{
node(POD_LABEL) {
stage("Pipeline job starts")
{
environment{
VERSION = "${env.BUILD_ID}"
}
stage("checkout scm") {
git branch: 'main', url: 'https://github.com/q-uest/CICD_Java_gradle_application'
}
stage("sonar uality check"){
script{
withSonarQubeEnv(credentialsId: 'sonarqube-id') {
sh 'chmod +x ./gradlew'
sh './gradlew sonarqube'
}
def qg = waitForQualityGate()
if (qg.status != 'OK') {
error "Pipeline aborted due to quality gate failure: ${qg.status}"
}
}
}
}
}
}