branch backups - juancamilocc/virtual_resources GitHub Wiki

Jenkins job for Branch Backups

This Jenkins job is designed to create backups of specific GitHub branches. The backup branches follow the naming format backup_<branch_name>-YYYY-MM-DD.

This Jenkins jobs performs the following actions.

  1. Clones the repository and checks if the specified branches exist.
  2. Creates backup branches for those that exist.
  3. Deletes old backup branches (older than three days).
  4. Sends notifications to Microsoft Teams upon success or failure.
pipeline {
    agent {
        kubernetes {
            cloud 'kubernetes-staging'
            yaml """
apiVersion: v1
kind: Pod
metadata:
  name: rocky-pod
  namespace: jenkins
spec:
  containers:
    - name: rocky
      image: ghcr.io/juancamilocc/builders:rocky8-docker
      imagePullPolicy: IfNotPresent
      command:
      - /busybox/cat
      tty: true
      securityContext:
        runAsUser: 0
        privileged: true
      resources:
        limits:
          memory: "2Gi"
          cpu: "900m"
        requests:
          memory: "1Gi"
          cpu: "500m"
    - name: jnlp
      image: jenkins/inbound-agent
      resources:
        limits:
          memory: "1Gi"
          cpu: "512m"
        requests:
          memory: "500Mi"
          cpu: "256m"
            """
        }
    }
    environment {
        REPOSITORY = "<repository>"
        DATE = sh(script: 'TZ="America/Bogota" date "+%Y-%m-%d"', returnStdout: true).trim()
        BRANCH_PROD = "<name_branch_prod>"
        BRANCH_STAGING = "<name_branch_staging>"
        BRANCH_PRE_PROD = "<name_branch_pre_prod>"
        BACKUP_BRANCH_PROD = "<name_backup_branch_prod>"
        BACKUP_BRANCH_STAGING = "<name_backup_branch_staging>"
        BACKUP_BRANCH_PRE_PROD = "<name_backup_branch_pre_prod>"
    }
    stages {
        stage("Branch Backups") {
            steps {
                container("rocky") {
                    script {
                        withCredentials([
                            usernamePassword(
                                credentialsId: 'Credentials-github-prod', 
                                usernameVariable: 'GIT_USERNAME', 
                                passwordVariable: 'GIT_PASSWORD'
                            )
                        ]) {
                            sh '''
                                git config --global --add safe.directory ${WORKSPACE}/<repository_name>
                                git config --global user.email "<your_email>"
                                git config --global user.name "<your_username>"
                                git clone https://$GIT_USERNAME:$GIT_PASSWORD@$REPOSITORY <repository_name>
                                cd <reposiory_name>/

                                # Backup branches
                                git fetch origin

                                BRANCHES=("$BRANCH_PROD" "$BRANCH_STAGING" "$BRANCH_PRE_PROD")
                                BACKUP_BRANCHES=("$BACKUP_BRANCH_PROD" "$BACKUP_BRANCH_STAGING" "$BACKUP_BRANCH_PRE_PROD")

                                for i in "${!BRANCHES[@]}"; do
                                    
                                    BRANCH="${BRANCHES[i]}"
                                    BACKUP="${BACKUP_BRANCHES[i]}-$DATE"
                                    
                                    if git ls-remote --exit-code --heads origin "$BRANCH"; then
                                        git checkout -b "$BACKUP" "origin/$BRANCH"
                                        git push origin "$BACKUP"
                                    else
                                        echo "Branch $BRANCH does not exist in origin."
                                    fi

                                done

                                echo "Veryfing old branches..."
                                # Get backup branches
                                git branch -r --format="%(refname:short)" | grep -oE 'backup_[^ ]*-[0-9]{4}-[0-9]{2}-[0-9]{2}' > backupBranches.txt
                            '''

                            def backupBranches = readFile('<repository_name>/backupBranches.txt').readLines()
                            
                            if (!backupBranches.isEmpty()) {

                                // Get timestamp current date
                                def currentDateTimestamp = sh(script: "date -d '$DATE' +%s", returnStdout: true).trim().toLong()

                                echo "Branches to review: ${backupBranches.join('\n')}"
                                
                                dir('<repository_name>') {
                                    backupBranches.each { branch ->
                                        echo "Validating branch ${branch}..."
                                        
                                        // Get date from branch in YYYY-MM-DD format
                                        def branchDateStr = branch.replaceAll(/.*-([0-9]{4}-[0-9]{2}-[0-9]{2})/, '$1')
                                        
                                        // Get timestamp of branch date
                                        def branchDateTimestamp = sh(script: "date -d '$branchDateStr' +%s", returnStdout: true).trim().toLong()
                                        
                                        // Calculate difference in days
                                        def diffInDays = (currentDateTimestamp - branchDateTimestamp) / 86400

                                        if (diffInDays > 3) {
                                            try {
                                                sh """
                                                    git fetch
                                                    git push origin --delete ${branch}
                                                """

                                            } catch (e) {
                                                echo "Failed to delete branch: ${branch} - ${e.message}"
                                            }
                                        } else {
                                            echo "Branch ${branch} isn't older than three days and won't be deleted."
                                        }
                                    }
                                }

                            } else {

                                echo "No backup branches found!"
                            }
                        }
                    }
                }
            }
        }
    }
    post {
        success {
            //Send teams notification
            withCredentials([string(credentialsId: '<teams_notifications_name>', variable: 'webhook')]) {
                office365ConnectorSend adaptiveCards: true, 
                color: 'good', 
                message: 'The weekly branch backups have been successful!', 
                status: 'SUCCESS', 
                webhookUrl: "${webhook}"    
            }
        }
        failure {
            //Send teams notification
            withCredentials([string(credentialsId: '<teams_notifications_name>', variable: 'webhook')]) {
                office365ConnectorSend adaptiveCards: true, 
                color: 'attention', 
                message: 'The weekly branch backups have failed!', 
                status: 'FAILURE', 
                webhookUrl: "${webhook}"    
            }
        }
    }
}

Let's explain each section of the Jenkins job.

First, we define the ephemeral container that will run the whole process, indicating docker image for rocky and jnlp, as well as resource parameters.

    agent {
        kubernetes {
            cloud 'kubernetes-staging'
            yaml """
apiVersion: v1
kind: Pod
metadata:
  name: rocky-pod
  namespace: jenkins
spec:
  containers:
    - name: rocky
      image: ghcr.io/juancamilocc/builders:rocky8-docker
      imagePullPolicy: IfNotPresent
      command:
      - /busybox/cat
      tty: true
      securityContext:
        runAsUser: 0
        privileged: true
      resources:
        limits:
          memory: "2Gi"
          cpu: "900m"
        requests:
          memory: "1Gi"
          cpu: "500m"
    - name: jnlp
      image: jenkins/inbound-agent
      resources:
        limits:
          memory: "1Gi"
          cpu: "512m"
        requests:
          memory: "500Mi"
          cpu: "256m"
            """
        }
    }

Here, define the environment variables like repository, date, branches and backup branches.

environment {
    REPOSITORY = "<repository>"
    DATE = sh(script: 'TZ="America/Bogota" date "+%Y-%m-%d"', returnStdout: true).trim()
    BRANCH_PROD = "<name_branch_prod>"
    BRANCH_STAGING = "<name_branch_staging>"
    BRANCH_PRE_PROD = "<name_branch_pre_prod>"
    BACKUP_BRANCH_PROD = "<name_backup_branch_prod>"
    BACKUP_BRANCH_STAGING = "<name_backup_branch_staging>"
    BACKUP_BRANCH_PRE_PROD = "<name_backup_branch_pre_prod>"
}

Set up git user details, clone the repository, get the latest branches, iterate over the branch list creating backups for those that exist, push the backup branches and create a file with a list of backup branches.

git config --global --add safe.directory ${WORKSPACE}/<repository_name>
git config --global user.email "<your_email>"
git config --global user.name "<your_username>"
git clone https://$GIT_USERNAME:$GIT_PASSWORD@$REPOSITORY <repository_name>
cd <reposiory_name>/

# Backup branches
git fetch origin

BRANCHES=("$BRANCH_PROD" "$BRANCH_STAGING" "$BRANCH_PRE_PROD")
BACKUP_BRANCHES=("$BACKUP_BRANCH_PROD" "$BACKUP_BRANCH_STAGING" "$BACKUP_BRANCH_PRE_PROD")

for i in "${!BRANCHES[@]}"; do
    
    BRANCH="${BRANCHES[i]}"
    BACKUP="${BACKUP_BRANCHES[i]}-$DATE"
    
    if git ls-remote --exit-code --heads origin "$BRANCH"; then
        git checkout -b "$BACKUP" "origin/$BRANCH"
        git push origin "$BACKUP"
    else
        echo "Branch $BRANCH does not exist in origin."
    fi

done

echo "Veryfing old branches..."
# Get backup branches
git branch -r --format="%(refname:short)" | grep -oE 'backup_[^ ]*-[0-9]{4}-[0-9]{2}-[0-9]{2}' > backupBranches.txt

Read the file with the backup branches, if the file is empty it does nothing but in case the file contains references it filters and deletes the old branches older than 3 days.

def backupBranches = readFile('<repository_name>/backupBranches.txt').readLines()
                            
if (!backupBranches.isEmpty()) {

    // Get timestamp current date
    def currentDateTimestamp = sh(script: "date -d '$DATE' +%s", returnStdout: true).trim().toLong()

    echo "Branches to review: ${backupBranches.join('\n')}"
    
    dir('<repository_name>') {
        backupBranches.each { branch ->
            echo "Validating branch ${branch}..."
            
            // Get date from branch in YYYY-MM-DD format
            def branchDateStr = branch.replaceAll(/.*-([0-9]{4}-[0-9]{2}-[0-9]{2})/, '$1')
            
            // Get timestamp of branch date
            def branchDateTimestamp = sh(script: "date -d '$branchDateStr' +%s", returnStdout: true).trim().toLong()
            
            // Calculate difference in days
            def diffInDays = (currentDateTimestamp - branchDateTimestamp) / 86400

            if (diffInDays > 3) {
                try {
                    sh """
                        git fetch
                        git push origin --delete ${branch}
                    """

                } catch (e) {
                    echo "Failed to delete branch: ${branch} - ${e.message}"
                }
            } else {
                echo "Branch ${branch} isn't older than three days and won't be deleted."
            }
        }
    }

} else {

    echo "No backup branches found!"
}

Finally, send notification via Teams according to the status Jenkins job.

post {
    success {
        //Send teams notification
        withCredentials([string(credentialsId: '<teams_notifications_name>', variable: 'webhook')]) {
            office365ConnectorSend adaptiveCards: true, 
            color: 'good', 
            message: 'The weekly branch backups have been successful!', 
            status: 'SUCCESS', 
            webhookUrl: "${webhook}"    
        }
    }
    failure {
        //Send teams notification
        withCredentials([string(credentialsId: '<teams_notifications_name>', variable: 'webhook')]) {
            office365ConnectorSend adaptiveCards: true, 
            color: 'attention', 
            message: 'The weekly branch backups have failed!', 
            status: 'FAILURE', 
            webhookUrl: "${webhook}"    
        }
    }
}

Conclusion

These backups play an important role in disaster recovery plans, ensuring a faster way to restore critical branches in case of accidental deletions, corruption, or unwanted changes. By automating the process, we reduce the risk of human error, maintain a structured backup history, and ensure that older backups are cleaned up efficiently.

⚠️ **GitHub.com Fallback** ⚠️