launch ephemeral jenkins agents - juancamilocc/virtual_resources GitHub Wiki

Launch ephemeral EC2 Agents using Jenkins and Terraform

In this guide, you will learn how to launch an ephemeral agent using Terraform and connect it to Jenkins as a node prepared to run demanding processes or jobs. This is ideal for parallel deployments and prevents overloading the main Jenkins controller.

Table of Content

General Workflow

General Workflow

The diagram above illustrates how Jenkins executes deployments using Terraform. This setup will include all the necessary configurations to launch an EC2 instance according to your requirements. It will connect via Jenkins port 50000, allowing traffic between the EC2 Jenkins Controller and the ephemeral EC2 agent instance.

Prerequisites

  • AWS account with permissions for EC2, VPC, IAM, S3, EKS, ECR, and CodeArtifact.
  • Terraform installed.
  • AWS CLI installed and configured.
  • Jenkins controller with port 50000 open and accessible.

Check tools on the EC2 Jenkins Controller

1. Check Jenkins Controller Port

Ensure port 50000 is mapped and open:

docker ps | grep jenkins
# Look for 0.0.0.0:50000->50000/tcp in the output

2. Install and check Terraform

sudo apt-get update && sudo apt-get install -y gnupg software-properties-common
wget -O- https://apt.releases.hashicorp.com/gpg | sudo gpg --dearmor -o /usr/share/keyrings/hashicorp-archive-keyring.gpg
echo "deb [signed-by=/usr/share/keyrings/hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(lsb_release -cs) main" | sudo tee /etc/apt/sources.list.d/hashicorp.list
sudo apt-get update && sudo apt-get install terraform
terraform --version
# Terraform v1.12.2
# on linux_amd64

3. Install and check aws cli

sudo apt-get install awscli
# aws --version
# aws-cli/1.22.34 Python/3.10.12 Linux/6.8.0-52-generic botocore/1.23.34

Create Jenkins Agent role

Log in to your AWS console, navigate to IAM > Roles > Create Role, and select Service or use case as EC2.

We must attach the following managed permissions.

  • AmazonEC2ContainerRegistryFullAccess
  • AmazonEKSClusterPolicy
  • AmazonS3FullAccess
  • AmazonSSMManagedInstanceCore
  • AWSCodeArtifactAdminAccess

Jenkins Agent role permissions

And the following custom inline policy:

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": [
                "eks:DescribeCluster",
                "sts:GetServiceBearerToken",
                "ecr:GetAuthorizationToken",
                "codeartifact:GetAuthorizationToken",
                "codeartifact:ReadFromRepository",
                "codeartifact:PublishPackageVersion",
                "codeartifact:DescribePackageVersion",
                "codeartifact:ListPackageVersions",
                "codeartifact:DescribeRepository",
                "codeartifact:PutPackageMetadata",
                "codeartifact:CreatePackageGroup"
            ],
            "Resource": "*"
        }
    ]
}

Jenkins Agent role custom policy

Prepare Agent Node in Jenkins

Go to your Jenkins controller, navigate to Manage Jenkins > Nodes > New Node, provide a name and create it as a Permanent Agent.

Create new node

In its configuration, set the number of executors to 8, remote root directory to /var/lib/jenkins_agent and the Launch method to Launch agent by connecting it to the controller.

Configuration node

You will notice we set 8 executors because we will use a t3a.2xlarge instance type, assigning each VCPU to one executor.

Finally, click on the node and Jenkins will show you how to connect the agent. We will use this information in the Terraform configuration.

Method connection node

NOTE: This will display the exposed URL for your Jenkins Controller and its secret.

To grant access to the EKS using the jenkins-agent-role, we must add permissions and include them in the aws-auth configmap, as follows.

rbac-permisions.yaml:

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: jenkins-agent-deployer
rules:
  # --- Rules for 'apps' API group ---
  - apiGroups: ["apps"]
    resources: ["deployments", "statefulsets", "daemonsets", "replicasets"] # Resources within the 'apps' group that these permissions apply to.
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # Verbs (actions) allowed on the specified resources.

  # --- Rules for core API group (empty string signifies core) ---
  - apiGroups: [""]
    resources: ["pods", "pods/log", "pods/exec", "services", "configmaps", "secrets", "persistentvolumeclaims", "events"] # Resources within the core group.
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # Verbs allowed on core resources.

  # --- Rules for 'batch' API group ---
  - apiGroups: ["batch"]
    resources: ["jobs", "cronjobs"] # Resources within the 'batch' group.
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # Verbs allowed on batch resources.

  # --- Rules for 'networking.k8s.io' API group ---
  - apiGroups: ["networking.k8s.io"]
    resources: ["ingresses"] # Resources within the 'networking.k8s.io' group.
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # Verbs allowed on networking resources.

  # --- Rules for 'autoscaling' API group ---
  - apiGroups: ["autoscaling"]
    resources: ["horizontalpodautoscalers"] # Resources within the 'autoscaling' group.
    verbs: ["get", "list", "watch", "create", "update", "patch", "delete"] # Verbs allowed on autoscaling resources.
---

apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: jenkins-agent-deployer-binding
subjects:
  # Defines the subjects (users, groups, or service accounts) that will be granted the role's permissions.
  - kind: Group 
    name: jenkins-agents
    apiGroup: rbac.authorization.k8s.io 
roleRef:
  kind: ClusterRole
  name: jenkins-agent-deployer
  apiGroup: rbac.authorization.k8s.io
kubectl apply -f rbac-permissions.yaml

And include it in the aws-auth, as follows.

kubectl -n kube-system get cm -o yaml aws-auth > aws-auth.yaml

Reference the permissions by adding the following lines.

apiVersion: v1
data:
  mapRoles: |
    .
    .
    .
    - groups:
      - jenkins-agents
      rolearn: arn:aws:iam::<YOUR_AWS_ID_ACCOUNT>:role/jenkins-agent-role
      username: jenkins-agent-user
  mapUsers: |
    - groups:
      - system:masters
      .
      .
      .
metadata:
  name: aws-auth
  namespace: kube-system
kubectl apply -f aws-auth.yaml

This configuration will grant full access to execute any action in the EKS cluster based on the role.

Terraform Configuration

  • Create an S3 bucket for the Terraform state:
aws s3 mb s3://terraform-backend-$(aws sts get-caller-identity --query "Account" --output text)
aws s3api put-bucket-versioning --bucket terraform-backend-<YOUR_AWS_ID_ACCOUNT> --versioning-configuration Status=Enabled
  • (Optional) Create an SSH key for debugging:
aws ec2 create-key-pair --key-name jenkins-agent-key --query "KeyMaterial" --output text > jenkins-agent-key.pem
chmod 400 jenkins-agent-key.pem

The project structure is as follows.

.
├── backend.tf
├── main.tf
├── modules
│   ├── ec2
│   │   ├── agent_config.sh
│   │   ├── main.tf
│   │   ├── outputs.tf
│   │   └── variables.tf
│   └── sg
│       ├── main.tf
│       ├── outputs.tf
│       └── variables.tf
└── variables.tf
  • backend.tf: Defines the backend configuration for Terraform state.
  • main.tf: Main configuration file, calls submodules.
  • modules/ec2/: EC2 instance definition and configuration.
  • modules/sg/: Security group definition.
  • variables.tf: Input variables for the project.

Let's define each file, as follows.

backend.tf:

terraform {
    backend "s3" {
        bucket         = "terraform-backend-<YOUR_AWS_ID_ACCOUNT>"
        key            = "terraform.tfstate"
        region         = "<YOUR_AWS_REGION>"
        encrypt        = true
        use_lockfile   = true
    }
}

main.tf:

provider "aws" {

    region  = var.aws_region
    profile = var.aws_profile
}


module "sg" {
    
    source                      = "./modules/sg"
    jenkins_controller_ip_cidr  = var.jenkins_controller_ip_cidr
    vpc_id                      = var.vpc_id
    ssh_ip_cidr                 = var.ssh_ip_cidr
}

module "ec2" {

    source                      = "./modules/ec2"
    ami_id                      = var.ami_id
    instance_type               = var.instance_type
    instance_name               = var.instance_name
    jenkins_controller_url      = var.jenkins_controller_url
    jenkins_agent_secret        = var.jenkins_agent_secret
    subnet_id                   = var.subnet_id
    key_name                    = var.key_name
    remote_fs_root              = var.remote_fs_root
    java_version                = var.java_version
    kubectl_version             = var.kubectl_version
    iam_instance_profile_name   = var.iam_instance_profile_name
    eks_cluster_name            = var.eks_cluster_name
    aws_region                  = var.aws_region
    aws_id                      = var.aws_id
    environment                 = var.environment
    sg_id                       = module.sg.sg_id
}

variables.tf:

# Variables backend and provider
variable "aws_region" {

    description = "AWS region"
    type        = string
    default     = "<YOUR_AWS_REGION>"
}

variable "aws_profile" {

    description = "AWS profile"
    type        = string
    default     = "default"
}

# Variables EC2 module
variable "ami_id" {

  description = "ID AMI for EC2 instance"
  type        = string
}

variable "instance_type" {

  description = "EC2 instance type"
  type        = string
  default     = "t3a.2xlarge"
}

variable "instance_name" {

  description = "EC2 instance name"
  type        = string
}

variable "jenkins_controller_url" {
  
  description = "Jenkins URL"
  type        = string
  default     = "<YOUR_JENKINS_CONTROLLER_URL>"
}

variable "jenkins_agent_secret" {

  description = "Generated secret for jenkins agent"
  type        = string
  sensitive   = true
}

variable "subnet_id" {
    
  description = "ID of the subnet where the EC2 instance will be launched."
  type        = string
  default     = "<SUBNET_TO_LAUNCH_YOUR_EC2_INSTANCE>"
}

variable "key_name" {

  description = "SSH Key to access to EC2"
  type        = string
  default     = "jenkins-agent-key"
}

variable "remote_fs_root" {

    description = "Jenkins Agent path where it will save agent file configuration"
    type        = string
    default     = "/var/lib/jenkins_agent"
}

variable "java_version" {

    description = "Java Version"
    type        = string
    default     = "11"
}

variable "kubectl_version" {

    description = "Kubectl version"
    type        = string
    default     = "v1.33.1"
}

variable "iam_instance_profile_name" {

    description = "Instance profile name of IAM role"
    type        = string
    default     = "jenkins-agent-role"
}

variable "eks_cluster_name" {

    description = "EKS cluster name"
    type        = string
    default     = "<NAME_EKS_CLUSTER>"
}

variable "aws_id" {

    description = "AWS id"
    type        = string
    default     = "<YOUR_AWS_ID_ACCOUNT>"
}

variable "environment" {

    description = "Environment context"
    type        = string
    default     = "testing"
}

# Variables sg module
variable "vpc_id" {

  description = "ID of the VPC where the security group will be created"
  type        = string
  default     = "<VPC_ID>"
}

variable "jenkins_controller_ip_cidr" {

  description = "CIDR block of the Jenkins controller for ingress rules"
  type        = string
  default     = "<CIDR_IP>"
}

variable "ssh_ip_cidr" {
    
  description = "CIDR block of the SSH key for ingress rules"
  type        = string
  default     = "<YOUR_IP_TO_USE_SSH_KEY>"
}

NOTE: The variables.tf file defines default values, you can replace them directly in the file or to use -var flag at execution time.

modules/sg/main.tf:

resource "aws_security_group" "executor_agent_sg" {

    name_prefix = "executor-agent-sg"
    vpc_id      = var.vpc_id

    # 50000 port is default to connect agents
    ingress {

        from_port   = 50000 
        to_port     = 50000
        protocol    = "tcp"
        cidr_blocks = [var.jenkins_controller_ip_cidr]
        description = "Allow Access from Jenkins to connect agent"
    }

    # SSH key access
    ingress {
        from_port   = 22
        to_port     = 22
        protocol    = "tcp"
        cidr_blocks = [var.ssh_ip_cidr] 
        description = "Allow SSH from my IP"
    }

    # Allow download any data from internet, to get software among others.
    egress {

        from_port   = 0
        to_port     = 0
        protocol    = "-1"
        cidr_blocks = ["0.0.0.0/0"]
    }
}

modules/sg/outputs.tf:

output "sg_id" {
    
    value = aws_security_group.executor_agent_sg.id
}

modules/sg/variables.tf:

variable "vpc_id" {
    
    description = "ID of the VPC where the security group will be created"
    type        = string
}

variable "jenkins_controller_ip_cidr" {
    
    description = "CIDR block of the Jenkins controller for ingress rules"
    type        = string
}

variable "ssh_ip_cidr" {
    
    description = "CIDR block of the SSH key for ingress rules"
    type        = string
}

modules/ec2/main.tf:

resource "aws_instance" "executor_agent" {
    
    ami                     = var.ami_id
    instance_type           = var.instance_type
    iam_instance_profile    = var.iam_instance_profile_name

    instance_market_options {
        
        market_type = "spot"
        
        spot_options {
            instance_interruption_behavior = "terminate"
        }
    }

    root_block_device {
    
        volume_size = 100  
        volume_type = "gp3" 
        delete_on_termination = true
    }

    subnet_id               = var.subnet_id
    vpc_security_group_ids  = [var.sg_id]
    key_name                = var.key_name

    # Pass variables for setting up instance as an agent and install requirements
    user_data = base64encode(templatefile("${path.module}/agent_config.sh", {
        
        jenkins_controller_url      = var.jenkins_controller_url
        jenkins_agent_name          = var.instance_name
        jenkins_agent_secret        = var.jenkins_agent_secret
        remote_fs_root              = var.remote_fs_root
        java_version                = var.java_version
        kubectl_version             = var.kubectl_version
        eks_cluster_name            = var.eks_cluster_name
        aws_region                  = var.aws_region
        aws_id                      = var.aws_id
        environment                 = var.environment
    }))

    tags = {
        Name = var.instance_name
    }
}

modules/ec2/agent_config.sh:

#!/bin/bash
set -euxo pipefail

# Install requirements
sudo apt update
sudo apt install -y \
  openjdk-${java_version}-jre-headless \
  default-jdk \
  maven \
  python3-pip \
  amazon-ecr-credential-helper \
  docker.io \
  git-all \
  jq \
  curl \
  unzip \
  apt-transport-https \
  ca-certificates \
  gnupg \
  software-properties-common

# Update CA certificates
sudo update-ca-certificates

# Install kubectl
curl -LO "https://dl.k8s.io/release/${kubectl_version}/bin/linux/amd64/kubectl"
sudo install -o root -g root -m 0755 kubectl /usr/local/bin/kubectl
rm -f kubectl 

# Install aws-cli
curl "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "awscliv2.zip"
unzip awscliv2.zip
sudo ./aws/install --update --bin-dir /usr/local/bin --install-dir /usr/local/aws-cli
rm -rf awscliv2.zip aws 

# Configure EKS access
sudo mkdir -p /home/ubuntu/.kube
sudo chown ubuntu:ubuntu /home/ubuntu/.kube
sudo -u ubuntu /usr/local/bin/aws eks update-kubeconfig --name ${eks_cluster_name} 
--region ${aws_region} --kubeconfig /home/ubuntu/.kube/config

# Install Terraform
wget -O- https://apt.releases.hashicorp.com/gpg | \
gpg --dearmor | \
sudo tee /usr/share/keyrings/hashicorp-archive-keyring.gpg > /dev/null
echo "deb [arch=$(dpkg --print-architecture) signed-by=/usr/share/keyrings/
hashicorp-archive-keyring.gpg] https://apt.releases.hashicorp.com $(grep -oP '(?
<=UBUNTU_CODENAME=).*' /etc/os-release || lsb_release -cs) main" | sudo tee /etc/apt/
sources.list.d/hashicorp.list
sudo apt update
sudo apt-get install terraform

# Install yq and update kubernetes context
sudo wget https://github.com/mikefarah/yq/releases/latest/download/yq_linux_amd64 -O /
usr/local/bin/yq
sudo chmod +x /usr/local/bin/yq
sudo -u ubuntu yq ".contexts[] |= select(.name == \"arn:aws:eks:${aws_region}:${aws_id}
:cluster/${environment}\") .name = \"${environment}\"" -i /home/ubuntu/.kube/config

# ECR configuration
sudo mkdir -p /home/ubuntu/.docker
sudo chown ubuntu:ubuntu /home/ubuntu/.docker
sudo -u ubuntu bash -c "cat <<EOF > /home/ubuntu/.docker/config.json
{
  \"credHelpers\": {
       \"${aws_id}.dkr.ecr.${aws_region}.amazonaws.com\": \"ecr-login\"
  }
}
EOF"

# Configuration as jenkins agent
sudo mkdir -p ${remote_fs_root}
sudo chown ubuntu:ubuntu ${remote_fs_root}

# Get JAR Jenkins Agent
sudo -u ubuntu curl -sS "${jenkins_controller_url}/jnlpJars/agent.jar" -o "$
{remote_fs_root}/agent.jar"

# Create script to execute jenkins agent
sudo -u ubuntu bash -c "cat <<EOF > ${remote_fs_root}/start_agent.sh
#!/bin/bash
java -jar "${remote_fs_root}/agent.jar" \
  -url "${jenkins_controller_url}/" \
  -secret "${jenkins_agent_secret}" \
  -name "${jenkins_agent_name}" \
  -workDir "${remote_fs_root}"
EOF"

# Give permissions
sudo -u ubuntu chmod +x "${remote_fs_root}/start_agent.sh"

# Create a service to execute Jenkins Agent
cat <<EOF > /etc/systemd/system/jenkins-agent.service
[Unit]
Description=Jenkins Agent
After=network.target

[Service]
ExecStart=${remote_fs_root}/start_agent.sh
User=ubuntu 
Group=ubuntu
Restart=always
RestartSec=10
StandardOutput=journal
StandardError=journal
WorkingDirectory=${remote_fs_root}

Environment="MAVEN_OPTS=-Djdk.tls.client.protocols=TLSv1.2 -Djdk.tls.server.enableSessionTicketExtension=false -Djdk.tls.client.enableSessionTicketExtension=false"

[Install]
WantedBy=multi-user.target
EOF

# Run and enable docker
sudo systemctl start docker
sudo systemctl enable docker
sudo usermod -aG docker ubuntu

# Run and enable Jenkins agent
sudo systemctl daemon-reload
sudo systemctl enable jenkins-agent
sudo systemctl start jenkins-agent

modules/ec2/outputs.tf:

output "ec2_instance_id" {
    
    value = aws_instance.executor_agent.id
}

modules/ec2/variables.tf:

variable "ami_id" {
    
    description = "ID of the AMI to use for the instance"
    type        = string
}

variable "instance_type" {
    
    description = "Type of instance to start"
    type        = string
}

variable "subnet_id" {
    
    description = "VPC Subnet ID to launch in"
    type        = string
}

variable "key_name" {
    
    description = "Key name of the Key Pair to use for the instance"
    type        = string
}

variable "instance_name" {
  
    description = "Name to be used on EC2 instance created"
    type        = string
}

variable "jenkins_controller_url" {
  
    description = "URL of the Jenkins controller"
    type        = string
}

variable "jenkins_agent_secret" {
  
    description = "Secret for the Jenkins agent"
    type        = string
    sensitive   = true
}

variable "remote_fs_root" {
  
    description = "Remote filesystem root for Jenkins agent"
    type        = string
}

variable "java_version" {

    description = "Java Version"
    type        = string
}

variable "sg_id" {

    description = "Security Group ID"
    type        = string
}

variable "kubectl_version" {

    description = "Kubectl version"
    type        = string
}

variable "iam_instance_profile_name" {

    description = "Instance profile name of IAM role"
    type        = string
}

variable "eks_cluster_name" {

    description = "EKS cluster name"
    type        = string
}

variable "aws_region" {

    description = "AWS region"
    type        = string
}

variable "aws_id" {

    description = "AWS id"
    type        = string
}

variable "environment" {

    description = "Environment context"
    type        = string
}

NOTE: As you can see, we are including an agent_config.sh file, to install different tools and establish the connection with the Jenkis controller server. This provides a recommended tools stac for CI/CD environments. Also, Environment="MAVEN_OPTS=-Djdk.tls.client.protocols=TLSv1.2 -Djdk.tls.server.enableSessionTicketExtension=false -Djdk.tls.client.enableSessionTicketExtension=false" is defined for older Java versions, ensuring connection with artifact repositories.

All content is in the repository Launch Epehemeral Agents using Jenkins and Terraform.

Create Jenkins Jobs to launch and delete the Ephemeral Agent

Go to your Jenkins Controller and create a pipeline with the following input parameters.

  • instance: Main node name to run the Jenkins job.
  • environment: Define your environment.
  • executeHeavyJob: Job name which runs a heavy process.
  • secretJenkinsAgent: Agent secret.
  • repository: Repository containing the Terraform code.
  • pathRepository: Path to the Terraform repository.

The Jenkinsfile is defined, as follows.

pipeline {
    agent {
        node {label "$instance"}
    }
    environment {
        AGENT_NAME = "executor_node"
        AMI_ID = getAMIID() 
    }
    stages{
        stage('Launch Executor Agent') {
            steps {
                script {
                    
                    agentReady = isAgentOnline(AGENT_NAME)

                    if (!agentReady) {

                        echo "The agent is not ready, lauching agent using Terraform..."

                        sh """
                            git clone ${repository} && cd ${pathRepository}
                            
                            set +x
                            terraform init
                            terraform plan \
                                -var="instance_name=${AGENT_NAME}" \
                                -var="jenkins_agent_secret=${secretJenkinsAgent}" \
                                -var="ami_id=${AMI_ID}" \
                                -out=jenkins-agent
                            terraform apply "jenkins-agent"
                        """

                        sh 'rm -rf $pathRepository'

                    } else {

                        echo "The agent is already active and online..."
                    }
                }
            }
        }
        stage('Check Agent Status') {
            steps {
                script {
                    
                    def MAX_RETRIES = 60             
                    def RETRY_DELAY_SECONDS = 10
                    def agentReady = false
                    def retries = 0

                    echo "Waiting for '${AGENT_NAME}' agent is active..."

                    while (!agentReady && retries < MAX_RETRIES) {
                        
                        agentReady = isAgentOnline(AGENT_NAME)

                        if (!agentReady) {
                            
                            retries++
                            echo "Retrying ${retries}/${MAX_RETRIES}: '${AGENT_NAME}' agent is not active. Waiting ${RETRY_DELAY_SECONDS} seconds..."
                            sleep RETRY_DELAY_SECONDS 
                        }
                    }

                    if (agentReady) {
                        
                        echo "The' ${AGENT_NAME}' agent is online and active..."
                    
                    } else {
                        
                        error "The '${AGENT_NAME}' agent is not active after ${MAX_RETRIES} attempts..."
                    }
                }
            }
        }
        stage('Launch Job heavy process') {
            steps {
                script {

                    // Execute job
                    build job: "${executeHeavyJob}", 
                        parameters: [
                            string(name: 'instance', value: "${AGENT_NAME}"),
                            string(name: 'secretJenkinsAgent', value: "${secretJenkinsAgent}"),
                            string(name: 'amiID', value: "${AMI_ID}")
                        ],
                        wait: false
                }
            }
        }
    }
    post {
        success {
            script {
                
                sendNotification(
                    "${AGENT_NAME} agent launched ✅",
                    "Executing redeployment the whole services with dependency of ${artifactLibrary}",
                    'N/A'
                )
            }
        }
        failure {
            script {

                sendNotification(
                    "${AGENT_NAME} agent failed ❌",
                    "Something was wrong launching the EC2 instance",
                    'Review compilation!'
                )
            }
        }
    }
}

def isAgentOnline(String agentName) {

    Node node = Jenkins.instance.getNode(agentName)
    if (node == null) {
        println("WARN: The '${agentName}' agent doesn't exist in Jenkins Server. Verify name...")
        return false 
    }

    Computer computer = node.toComputer()
    if (computer == null) {
        println("WARN: Computer agent not found to the agent '${agentName}'.")
        return false
    }

    if (computer.isOffline() || computer.isTemporarilyOffline() || computer.isConnecting
    ()) {
        println("'${agentName}' agent is offline. Current status: ${computer.isOffline() ? 'OFFLINE' : (computer.isTemporarilyOffline() ? 'TEMPORALMENTE OFFLINE' : 'CONECTANDO')}. Cause: ${computer.getOfflineCause() ?: 'Any'}")
        return false
    }

    return true
}

def getAMIID() {

    def amiID = sh(
        script: 'aws ssm get-parameter \
            --name /aws/service/canonical/ubuntu/server/22.04/stable/current/amd64/hvm/ebs-gp2/ami-id \
            --query "Parameter.Value" \
            --output text',
        returnStdout: true
    ).trim()

    return amiID
}

def sendNotification(String status, message, pendingActions) {
    
    withCredentials([
        string(
            credentialsId: 'webhookUrlGoogle', 
            variable: 'webhook'
        )
    ]) {
        configFileProvider([
            configFile(fileId: 'launch-agent-notification', variable: 'notifications')
        ]) {

            def date = sh(script: 'TZ="America/Bogota" date "+%Y-%m-%d-%H-%M-%S"', returnStdout: true).trim()
            def requestNotification = readFile(notifications)

            requestNotification = requestNotification.replace('${date}',                    "${date}")
            requestNotification = requestNotification.replace('${environment}',             "${environment}")
            requestNotification = requestNotification.replace('${projectName}',             "${currentBuild.projectName}")
            requestNotification = requestNotification.replace('${absoluteUrl}',             "${currentBuild.absoluteUrl}")
            requestNotification = requestNotification.replace('${status}',                  status)
            requestNotification = requestNotification.replace('${message}',                 message)
            requestNotification = requestNotification.replace('${pendingActions}',          pendingActions)

            writeFile file: 'payloadNotification.json', text: requestNotification
            sh 'curl -X POST -H "Content-Type: application/json" -d @payloadNotification.json $webhook' 
        }
    }
}

According to the Jenkinsfile above, it launches the agent using Terraform, validates every 10 seconds until the agent is online and active, executes a job with a heavy process and sends a notification.

For notifications, navigate to Manage Jenkins > Plugins > Available plugins, search Config File Provider and install it. This allows you to save files in a centralized manner.

Go to Manage Jenkins > Managed files > Add a new Config > Json file give it an id launch-agent-notification with the following content.

{
    "cardsV2": [
        {
            "cardId": "build-status",
            "card": {
                "header": {
                    "title": "${status}",
                    "subtitle": "Project Jenkins Pipeline",
                    "imageUrl": "https://lh6.googleusercontent.com/proxy/6lnHnmTlMw7ikRbvUJ4gZCPQhOp9YSL-7IUSxCpCzk3vRD35-O6fxTqYkkoRdTdEdu1y6iIU2wzz6w9gf-Z3Lr6MDPf3L3e048MSe02ehCHm2lKqGPUJSfg0W-JMjggipSapB_6KYFKGYkXanSJwsIuaDz3C-LvMXRAbE8DnogXSsu7qjXYQNjY8LzQgzPF-WcKWkfZzbrt4ni1xIqKw20tGKeCanb8k",
                    "imageType": "CIRCLE"
                },
                "sections": [
                    {
                        "widgets": [
                            {
                                "decoratedText": {
                                    "text": "<b>Date:</b> ${date}"
                                }
                            },
                            {
                                "decoratedText": {
                                    "text": "<b>Environment:</b> ${environment}"
                                }
                            },
                            {
                                "decoratedText": {
                                    "text": "<b>Project:</b> ${projectName}"
                                }
                            },
                            {
                                "decoratedText": {
                                    "text": "<b>Message:</b> ${message}",
                                    "wrapText": true
                                }
                            }, 
                            {
                                "decoratedText": {
                                    "text": "<b>Pending Actions:</b> ${pendingActions}",
                                    "wrapText": true
                                }
                            },
                            {
                                "decoratedText": {
                                    "text": "<b>Author:</b> DevOps Team",
                                    "startIcon": {
                                        "knownIcon": "PERSON"
                                    }
                                }
                            },
                            {
                                "buttonList": {
                                    "buttons": [
                                        {
                                            "text": "View Build",
                                            "onClick": {
                                                "openLink": {
                                                    "url": "${absoluteUrl}"
                                                }
                                            }
                                        }
                                    ]
                                }
                            }
                        ]
                    }
                ]
            }
        }
    ]
}

Notification Managed Files

Define the Jenkinsfile that will run a heavy process, as follows.

pipeline {
    agent {
        node {label "$instance"}
    }
    stages{
        stage('Get and Execute Jobs') {
            steps {
                script {

                    // logic heavy process...
                }
            }
        }
    }
    post {
        success {
            echo "SUCESS..."
            // Logic success post action
        }
        unstable {
            echo "UNSTABLE..."
            // Logic unstable post action
        }
        failure {
            echo "FAILUE..."
            // Logic failure post action
        }
        cleanup {
            script {

                echo "Deleting Agent Instance..."

                try {
                    retry(3) {

                        sh """
                            git clone -b ${repositoryTF} && cd ${pathRepositoryTF}
                    
                            set +x
                            terraform init
                            terraform destroy \
                                -var="instance_name=${agentNameTF}" \
                                -var="jenkins_agent_secret=${secretJenkinsAgent}" \
                                -var="ami_id=${amiID}" \
                                -auto-approve
                        """

                        sh 'rm -rf $pathRepositoryTF'
                    }

                } catch (Exception e) {

                    echo "Something was wrong destroying EC2 agent..."
                    throw e
                }
            }
        }
    }
}

NOTE: As you will notice, we are using internal Jenkins functions, so you will get warnings like these.

Approve exceptions for pipelines

To solve this, you must approve them manually by clicking approve.

Approve exceptions for pipelines

You should accept all signatures until they are approved, similar to this.

Signatures approved as admin

IMPORTANT: It's mandatory to accept all of them, since they are used in functions to check the agent status, filter jobs, and other operations.

Conclusions

This guide provides a comprehensive solution for dynamically provisioning and managing ephemeral EC2 Jenkins agents using Terraform. By automating the creation, configuration, and termination of these agents, you can significantly enhance your CI/CD pipeline's efficiency, scalability, and cost-effectiveness. This approach ensures that your Jenkins controller remains unburdened while heavy workloads are handled by dedicated, on-demand resources, ultimately leading to faster and more reliable deployments.

⚠️ **GitHub.com Fallback** ⚠️