Jenkins SSH Pipeline Steps Plugin - chaitanyavangalapudi/devops-scripts GitHub Wiki

Jenkins SSH Pipeline Steps plugin is very useful if you want to connect to a remote Linux machine and execute commands via Jenkins. This requires installation of two SSH related jenkins plugins Credentials Plugin, Credentials Binding Plugin plugins as prerequisites. We will discuss how to use SSH Pipeline Steps Plugin in Jenkins Declarative pipeline to deploy binaries to remote Deployment Box.

Step#1 Install and Configure SSH Credentials Plugins

To add ssh credentials on Jenkins server UI, we need to have Credentials Plugin and Credentials Binding Plugin installed on Jenkins server. Log in to Jenkins and navigate to System > Manage Jenkins > Manage Plugins. Search for these two plugins in Available Plugins section, Install them and Restart Jenkins to make sure installations come into effect.

Step#2 Install SSH Steps Plugin

Follow the above procedure and install SSH Pipeline Steps plugin, and restart Jenkins.

Step#3 Setup communication with remote host via SSH

You need to create a public/private key pair as the Jenkins user on your Jenkins server, then copy the public key to the user you want to do the deployment with on your target server. The private key needs to be pasted in the Jenkins UI.

Step#3.1 Generate SSH Public and Private Key Pair on Jenkins machine

ssh-keygen -f ~/.ssh/jenkins-privatekey-rsa -o -t rsa -b 4096 -C "Jenkins SSHkey"

Type a strong passphrase and retype, Remember the passphrase for later use. Let us assume passphrase is "passphrase2019".

This will generate two files jenkins-privatekey-rsa (Private Key), jenkins-privatekey-rsa.pub (Public Key)

Step#3.2 Copy SSH Public Key file contents to Remote (Deployment) Box

Login to Remote machine as deployment user, and go to ** ~/.ssh** directory, open/create authorized_keys file and append the contents of jenkins-privatekey-rsa.pub file on the Jenkins box.

scp ~/.ssh/jenkins-privatekey-rsa.pub deployment@remotehost:~/
ssh deployment@remotehost
cd .ssh
cat ~/jenkins-privatekey-rsa.pub >> ~/.ssh/authorized_keys

Step#3.3 Change permissions of authorized_keys on remote box

Make sure your .ssh directory has permissions 700 and your authorized_keys file has permissions 600 by Running below commands

chmod go-w ~/
chmod 700 ~/.ssh
chmod 600 ~/.ssh/authorized_keys

Step#4 Configure Credentials on Jenkins UI

Goto Credentials >> Add Credentials >> SSH Username with private key on Jenkins UI and add below configuration

Select Scope as Global - so that this can be used by jobs
Enter ID - JenkinsDeploymentKeyCred
Description - Deployment Private Key Credentials for Jenkins User
Username - Jenkins
Private Key - Enter Directly - Copy paste the contents of Jenkins Private key (/home/Jenkins/.ssh/jenkins-privatekey-rsa) into the Text Area
Passphrase - Enter passphrase2019 that you used earlier while creating the SSH key pair

Note: SSH Credentials Plugin no longer supports SSH credentials from files on the Jenkins master file system, neither user-specified file paths nor ~/.ssh. Existing SSH credentials of these kinds are migrated to "directly entered" SSH credentials. Refer link https://jenkins.io/security/advisory/2018-06-25/#SECURITY-440 for more details.

Step#5 Configure SSH Server on Jenkins UI

Go to Manage Jenkins >> Configure System >> SSH Servers and Add Remote server details

Name - deploy.mycompany.com
HostName - deploy.mycompany.com
UserName - deployment
Remote Directory - /

Under "Advanced" Configuration Enter below information

Select Use password authentication, or use a different key

Enter Passphrase / Password: passphrase2019
Key - Paste the contents of Jenkins Private key (/home/Jenkins/.ssh/jenkins-privatekey-rsa) into the Text Area

Using SSH Pipeline Steps Plugin in Jenkins Declarative Pipeline

Step #1 Discussion about Credentials Binding Plugin

As discussed in https://jenkins.io/doc/pipeline/steps/credentials-binding/ , this plugin supports various types of Credentials like usernameColonPassword, sshUserPrivateKey, string, AmazonWebServicesCredentialsBinding etc and it allows various kinds of credentials (secrets) to be used in idiosyncratic ways. Each binding will define an environment variable active within the scope of the step. You can then use them directly from any other steps that expect environment variables to be set.

Note:

The credentials will only be visible within the block passed to withCredentials, not outside of that.
There is no way to make the credentials globally available without switching to Scripted Pipeline. With Scripted, you can wrap your entire job in withCredentials. This is not possible with Declarative.

In our case, we will concentrate on sshUserPrivateKey type of credential.

Step #2 Discussion about sshUserPrivateKey Credentials

Copies the SSH key file given in the credentials to a temporary location, then sets a variable to that location. (The file is deleted when the build completes.) Also optionally sets variables for the SSH key's username and passphrase. Warning: if the master or slave node has multiple executors, any other build running concurrently on the same node will be able to read the contents of this file.

keyFileVariable: Name of an environment variable to be set to the temporary path of the SSH key file during the build.
credentialsId: Credentials of an appropriate type to be set to the variable.
passphraseVariable (optional): Name of an environment variable to be set to the password during the build. (optional)
usernameVariable (optional): Name of an environment variable to be set to the username during the build. (optional)

Sample Skeleton Jenkins Declarative Pipeline using this Credentials Binding plugin:

pipeline {
    agent {
        // define agent details
    }
    stages {
        stage('SSH Stage 1') {
            steps {
                withCredentials(bindings: [sshUserPrivateKey(credentialsId: 'JenkinsPrivateKey1',
                                                             keyFileVariable: 'SSH_KEY_FOR_DEPLOYMENT')]) {
                  // 
                }
                withCredentials(bindings: [certificate(credentialsId: 'JenkinsPrivateKey2',
                                                       keystoreVariable: 'CERTIFICATE_FOR_REMOTE', \
                                                       passwordVariable: 'XYZ-CERTIFICATE-PASSWORD')]) {
                  // 
                }
            }
        }
        stage('Stage 2') {
            steps {
                // 
            }
        }
    }
}

Let us discuss this in depth by taking below code snippet as an example

 withCredentials([sshUserPrivateKey(credentialsId: 'JenkinsDeploymentKeyCred',
                     keyFileVariable: 'JenkinsPrivateKey',
                     passphraseVariable: 'PASSPHRASE',
                     usernameVariable: 'USERNAME')]) {

     // use date for tag
     def tag = new Date().format("yyyyMMddHHmm")

     sh 'echo UserName=$USERNAME PassPhrase=$PASSPHRASE'
     sh 'echo UserName=${USERNAME} PassPhrase=${PASSPHRASE} TAG:${tag}'
     sh 'echo UserName=${env.USERNAME} PassPhrase=${env.PASSPHRASE}'
     println(env.USERNAME)
     sh "echo 'UserName=$USERNAME \n PassPhrase=$PASSPHRASE \n\n'"
 }

In this case, We need to use the JenkinsDeploymentKeyCred credential that we created in Jenkins UI which is of type "SSH Username with private key" as credentialsId. This will automatically bind the fields which are part of the credentials to variables that can be accessed in our pipeline steps. In our case

Private key for server is stored in Jenkins with credential id ‘JenkinsPrivateKeyCred’ and can be accessible via variable ‘JenkinsPrivateKey’
Passphrase that we entered (passphrase2019) is accessible via ‘PASSPHRASE’
UserName is accessible via ‘USERNAME’

Step #3 Using SSH Pipeline Steps Plugin

Step #3.1 Populating the remote variable used for SSH connection

As discussed in https://github.com/jenkinsci/ssh-steps-plugin#examples , most of the steps in this plugin require a common step variable called remote, which is Map of remote node settings such as user name, password and so on.

Since we are using public-key authentication, we will use keyFileVariable, passphraseVariable, usernameVariable for our Remote connection, apart from HostName. For additional security, we went with passphrase for our ssh key generation and we will use that in our case.
Rather than using "Private key for public-key authentication", we will use "Private key file name for public-key authentication". This will be used as identityFile for our remote connection

To Summarize, we will add below Fields to the remote variable

remote.host - HostName of remote deployment machine
remote.user - USERNAME
remote.passphrase - PASSPHRASE
remote.identityFile - JenkinsPrivateKey
remote.fileTransfer - File transfer method, that is SFTP or SCP. Defaults to SFTP. We will leave this as it is.

Step #3.2 Running the SSH operations

The plugin supports various operations such as

sshCommand - This step executes given command on remote node and responds with output.
sshScript - This step executes given script(file) on remote node and responds with output.
sshPut - Put a file or directory into the remote host.

Common Options:

remote variable which represents the Host config to run the command on is Mandatory for all these operations.
failOnError option is also common across operations, with default value "true", If this is false, no job failure would occur though there is an error while running the command.

Sample Jenkins declarative pipeline using these operations is:

pipeline {
   agent any
   environment {
        DIST_ARCHIVE_VAR="dist.${JOB_NAME}-${BUILD_ID}.tar.gz"
   }
   stages {
    stage('Deploy Using SSH Steps') {
      steps {
       script {
          def remote = [:]
          remote.name = "deployment.host.com"
          remote.host = "deployment.host.com"
          remote.allowAnyHosts = true

          def fileName = "${DIST_ARCHIVE_VAR}"
          def fileNameReplaced = fileName.replaceAll(/\-/,'\\\\-')
          echo fileName
          echo fileNameReplaced
          def command1 = "mv !(${fileName}|backup_dir) backup_dir || true"
          def command2 = "mv /app/dist/*.gz /app/dist/backup_dir || true"

          withCredentials([sshUserPrivateKey(credentialsId: 'JenkinsPrivateKey', 
                    keyFileVariable: 'JENKINS_PRIVATE_KEY', passphraseVariable: 'PASSPHRASE',
                    usernameVariable: 'USERNAME')]) {
             remote.user = USERNAME
             remote.passphrase = PASSPHRASE
             remote.identityFile = JENKINS_PRIVATE_KEY
             sshCommand remote: remote, command: 'mkdir -p /app/test/backup', failOnError: 'true'
             sshPut remote: remote, from: fileName, into: '/app/test/temp'
             sshCommand remote: remote, command: '/usr/sbin/fuser -k 9999/tcp || true', failOnError: 'true'
             sshCommand remote: remote, command: command2, failOnError: 'true'
             sshCommand remote: remote, command: command1, failOnError: 'true'
          }
       }
     }
   }
  }
}

Note:

It is observed that when you run "cd /app/abcd/xyz", the plugin doesn't change to the directory on your remote machine. So it is advisable to use the absolute paths on the remote machine.
If you don't want your build to fail when there is a failure on a command you can either add "|| true" to your command OR set failOnError: 'false'
If your fileName has any special characters like -, since Linux treats it as special character for options, we need to escape it by adding \ before that. If you use scripted pipeline with groovy, you need to use three slashes (\) in place of single slash to escape -. This is required for a case you want to move or copy one or some of the files using REGEX as shown above by adding shopt -s extglob option to your .bashrc
Since - has special meaning for a regexp, and it is easily overseen that the first argument to this method call is a string representing a regexp. - is used to represent range ([0-9]), therefore you have to escape it. So the right text should be

myText.replaceAll('\\-',replace);

myText.replaceAll("\\\-",replace);

(three \ because you have to escape the $ in Groovy too, to avoid it becomming a GString)

If you want to run any script which is present on the remote machine use sshCommand

sshCommand remote: remote, command: './deploy.sh', failOnError: 'true'

If you want to execute script present in your workspace on Jenkins server, you can use sshScript

sshScript remote: remote, script: "deploy.sh"

If the script is not present in the workspace, Jenkins errors out

java.lang.IllegalArgumentException: /home/jenkins/jobs/myproject-pipeline/workspace/deploy.sh does not exist.

References: