5 2 3 - Shippable/support GitHub Wiki

5.2.3 Release Notes

Release Date

8:00pm PST, Friday, Feb 17, 2017

Features

Ability to configure CI Projects as Pipeline jobs A new resource ciRepo and a new job runCI allow configuring CI projects as Pipeline jobs. Check out the tutorial for connecting CI to Pipelines
Integrations added to cliConfig resource: Docker Trusted Registry, JFrog Artifactory, Private Docker Registry, Terraform and Packer have been added to the existing list of supported integrations
More environmental variables in runSh and runCLI jobs: The following variables JOB_PATH, JOB_STATE, JOB_PREVIOUS_STATE, JOB_MESSAGE, JOBNAME_NAME, JOBNAME_TYPE, JOBNAME_PATH,JOBNAME_STATE, JOBNAME_PREVIOUS_STATE, JOBNAME_MESSAGE, RESOURCENAME_STATE and RESOURCENAME_META are added. Read more in our documentation
All subscriptions running on Shippable Nodes have an updated Shippable Agent: We made changes to the Shippable agent for improved performance. Read more about it in our documentation.
Custom Nodes reset to implement the updated Shippable Agent: A reset was enforced for all custom nodes that were not reset since the 5.2.2 release. Builds will fail on custom nodes initialized using a script, if they have not been reset. For more details, read the FAQ section on this issue
Console logs stored for three months: With this release, we've increased the duration of storing the console logs. Read details in our documentation
OAuth login integration with GitLab EE: This feature can be enabled in system integrations for Server customers only

Server installer changes

Removing a 'System integration' from the state.json file will remove the corresponding integration
Drydock tag added to systemMachineImages
Installer runs database clean up after provisioning services

Fixes

Builds failed for few users upon resetting their custom nodes: This fix ensures the correct packages for binaries are loaded and the Docker Hub images have correct versions. Builds will run successfully after a custom node reset
CI jobs fail when trying to pull Docker images: Jobs failed due to a Docker DNS issue. This fix adds a nameserver to the stable/unstable AMIs and resolves the issue
Limits console size to 64 MB: With this limit, if the console size exceeds 64 MB, you'll see the error Console size exceeds 64 MB limit. Truncated from here.
Escaping of CI environment variables is inconsistent and causes build failures: We are now escaping more special characters for CI environment variables
jobConsole download file name is too generic: Filename is now in format org-project-runNumber.jobNumber.log
Only Project owners can synchronize a project: The fix allows super users and non-owners to synchronize a project
Super users unable to access Account used to Process Webhooks: Fixes Super User access to Account Used to Process Webhooks panel in 'Project/Settings/runsConfig' section in the dashboard
API process does not retry a connection after a disconnect from RabbitMQ: The fix ensures the API process ends when there's a disconnect from RabbitMQ. This notifies Swarm to bring up a new instance of the API process and reconnect to RabbitMQ

Upgrading Server from 5.2.2 to 5.2.3

To upgrade Shippable Server, follow the instructions below. All steps in the upgrade happen on the Swarm Master machine except when noted otherwise.

Pre-installation:

Backup the usr/state.json file and the Postgres database (on the core machine)
Pull the latest code by running the following:
- git fetch
- git tag
- git checkout v5.2.3
Remove the following integrations from masterIntegrations array in usr/state.json file
- Git store
- SMTP
- Email
- mailgun
- braintree
- S3
- github (type auth)
- bitbucket (type auth)
- githubenterprise (type auth)
- bitbucketServer (type auth)
- AWS (type cloudproviders)
- hubspot (type mktg)
Remove the following integrations from systemIntegrations array in usr/state.json file
- Email NF
- braintree
- s3
- hubspot
- gitab (masterName=Git store)
- AWS-ROOT
- github.com (masterType: auth)
- Email NF 22
- bitbucket.com (masterType: auth)
- SMTP

Edit usr/state.json file to add the following integrations

```
[
    {   
      "type": "generic",
      "name": "hubspotToken"
    },
    {   
      "type": "generic",
      "name": "bitbucketserverKeys"
    },
    {   
      "type": "generic",
      "name": "githubenterpriseKeys"
    },
    {   
      "type": "generic",
      "name": "bitbucketKeys"
    },
    {   
      "type": "generic",
      "name": "githubKeys"
    },
    {   
      "type": "generic",
      "name": "braintreeKeys"
    }
    {
      "type": "generic",
      "name": "mailgunCreds"
    }
]
```

Edit the usr/state.json file to add the following systemIntegrations array

```
[
   {
     "name": "mktg",
     "masterName": "hubspotToken",
     "formJSONValues": [
       {
         "label": "hubspotApiEndPoint",
         "value": "<copy value from 'hubspot' system integration>"
       },  
       {
         "label": "hubspotApiToken",
         "value": "<copy value from 'hubspot' system integration>"
       }
     ]
   },
   {
     "formJSONValues": [
       {
         "value": "<copy value from 'braintree' system integration>",
         "label": "braintreeMerchantId"
       },
       {
         "value": "<copy value from 'braintree' system integration>",
         "label": "braintreePrivateKey"
       },
       {
         "value": "<copy value from 'braintree' system integration>",
         "label": "braintreeEnvironment"
       },
       {
         "value": "<copy value from 'braintree' system integration>",
         "label": "braintreePublicKey"
       }
     ],
     "masterName": "braintreeKeys",
     "name": "payment"
   },
   {
     "name": "auth",
     "masterName": "bitbucketserverKeys",
     "formJSONValues": [
       {
         "value": "<copy value from 'bitbucketServer' system integration>",
         "label": "clientSecret"
       },
       {
         "value": "<copy value from 'bitbucketServer' system integration>",
         "label": "clientId"
       },
       {
         "value": "<copy value from 'bitbucketServer' system integration>",
         "label": "url"
       },
       {
         "value": "<copy value from 'bitbucketServer' system integration>",
         "label": "wwwUrl"
       }
     ]
   }
   {
     "name": "auth",
     "masterName": "githubenterpriseKeys",
     "formJSONValues": [
       {
         "value": "<copy value from 'GHE' system integration>",
         "label": "clientSecret"
       },
       {
         "value": "<copy value from 'GHE' system integration>",
         "label": "clientId"
       },
       {
         "value": "<copy value from 'GHE' system integration>",
         "label": "url"
       },
       {
         "value": "<copy value from 'GHE' system integration>",
         "label": "wwwUrl"
       }
     ]
   },
   {
     "name": "auth",
     "masterName": "githubKeys",
     "formJSONValues": [
       {
         "label": "clientId",
         "value": "<copy value from 'github.com' system integration>"
       },
       {
         "label": "clientSecret",
         "value": ""<copy value from 'github.com' system integration>""
       },
       {
         "label": "wwwUrl",
         "value": ""<copy value from 'github.com' system integration>""
       },
       {
         "label": "url",
         "value": ""<copy value from 'github.com' system integration>""
       }
     ]
   },
   {
     "name": "auth",
     "masterName": "bitbucketKeys",
     "formJSONValues": [
       {
         "label": "clientId",
         "value": "<copy value from 'bitbucket.org' system integration>"
       },
       {
         "label": "clientSecret",
         "value": ""<copy value from 'bitbucket.org' system integration>""
       },  
       {
         "label": "wwwUrl",
         "value": ""<copy value from 'bitbucket.org' system integration>""
       },  
       {
         "label": "url",
         "value": ""<copy value from 'bitbucket.org' system integration>""
       }
     ]
   },
   {
     "name": "provision",
     "masterName": "amazonKeys",
     "formJSONValues": [
      {
        "label": "accessKey",
        "value": "<copy value from 'AWS-ROOT' system integration>"
      },  
      {
        "label": "secretKey",
        "value": "<copy value from 'AWS-ROOT' system integration>"
      }
     ]
   },
   {
     "name": "notification",
     "masterName": "smtpCreds",
     "formJSONValues": [
       {
         "label": "emailAuthPassword",
         "value": "<Copy email auth password from smtp>"
       },
       {
         "label": "emailAuthUser",
         "value": "<Copy email auth user from smtp>"
       },
       {
         "label": "emailSender",
         "value": "<Copy email sender from smtp>"
       },
       {
         "label": "host",
         "value": "<copy host from smtp>"
       },
       {
         "label": "port",
         "value": "<copy port from smtp>"
       },
       {
         "label": "secure",
         "value": "false"
       }
     ]
   }

] ```

For using notifications using mailgun insert the following systemIntegrations

  {
      "formJSONValues": [
        {
          "value": "<copy value from 'Mailgun NF 22' system integration>",
          "label": "apiKey"
        },
        {
          "value": "<copy value from 'Mailgun NF 22' system integration>",
          "label": "domain"
        }
      ],
      "masterName": "mailgunCreds",
      "name": "notification"
    }

Add "drydockTag": "prod" to existing "systemMachineImages"
Remove dynamicNodesSystemIntegrationId field from the usr/state.json file
Manually clean up S3 artifact systemIntegrations from DB/Vault and the usr/state.json file
Manually create a directory named logs in base/usr
Add a field systemSettings.nodeScriptsRemoteLocation="https://github.com/Shippable/node/tarball" in usr/state.json
Run sudo apt-get update on all machines
Enable/disable any integrations by adding/removing services in the systemIntegration section of the state.json file

Installation:
- Run sudo ./base.sh --release v5.2.3
Post installation:
- Stop the genexec/mexec container manually before resetting the systemNodes
- Reset all system nodes
- Verify genexec/mexec are running the latest version