Home - Jimmy-Xu/jenkins-hypercli-image GitHub Wiki

How to integrate Jenkins with Hyper_

Use Case

Project source code is hosted on GitHub
Jenkins server(with hyper cli installed) is hosted on Hyper_
Use Jenkins job to build(pull,make,test and upload) project in container which hosted on Hyper_
Start Jenkins build when a change is pushed to GitHub
When Jenkins start build, a new Hyper_ container will be created
The whole build process(pull code-> unit-test-> make-> integration-test-> upload) will be done in this Hyper_ container
If Jenkins build successfully, the result(binary archive file) will be uploaded to AWS S3
Finally, the Hyper_ container will be destroy

Solution

There are two ways to integrate Jenkins with Hyper_

Jenkins + hyper-plugin ( require hyper api for java, not ready )
Jenkins + hyper cli ( require hyper command line )

This demo will show the 2nd way.

Here is the diagram

assets/jenkins_with_hypercli.PNG

Prepare

Account

Example project

jenkins-hyper-example-app

Description:
- this is a golang example project, output a binary file "reverse"
  - usage： ./reverse hello world, output dlrow olleh
- this project will be build(pull code-> unit-test-> make-> integration-test-> upload) in a container with a golang env
- there is a pre-build image( hyperhq/jenkins-hyper-example-builder ) for build this project
Source Code(public repo): https://github.com/hyperhq/jenkins-hyper-example-app

Docker image

Jenkins server
- The image hyperhq/jenkins-hypercli:2.10 is base on jenkinsci/jenkins:2.10.
- There is a hyper cli installed in it(use hyper cli to run temporary container to build project).
- This is the Dockerfile.
Build env for golang project
- This image hyperhq/jenkins-hyper-example-builder:latest ) is base on golang:1.5.
- There is a awscli installed in it(use aws s3 cli to upload binary archive).
- This is the Dockerfile.

Step by Step

Step 1. Run jenkins server on Hyper_

Require:
- Hyper_ credential
- hyper cli
- image on docker hub: hyperhq/jenkins-hypercli:2.10

Run the following command in your local HostOS:

// allocate public ip
$ hyper fip allocate 1
23.236.114.227

// create volume
$ hyper volume create --name jenkins-data

// run jenkins container(use Hyper_ Credential here)
$ ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx
$ SECRET_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx
$ hyper run -d --name jenkins-hyper-demo \
    --size=l \
    -v jenkins-data:/var/jenkins_home \
    -e ACCESS_KEY=$ACCESS_KEY \
    -e SECRET_KEY=$SECRET_KEY \
    hyperhq/jenkins-hypercli:2.10

// associate public ip to jenkins container
$ hyper fip associate 23.236.114.227 jenkins-hyper-demo

// test hypercli in jenkins container(show tenantid of Hyper_)
$ hyper exec -it jenkins-hyper-demo hyper info | grep ID
ID: 56bfa4e8063c4872axxxxxxxxxxxxxxx

// get Jenkins unlock password
$ hyper logs jenkins-hyper-demo
...
*************************************************************
*************************************************************
*************************************************************
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:

e85299cb69274de48f5a09xxxxxxxxxxxxxx
...

$ hyper ps
CONTAINER ID   IMAGE                           COMMAND                  CREATED        STATUS        PORTS                        NAMES               PUBLIC IP
60aaf8d60c4d   hyperhq/jenkins-hypercli:2.10   "/bin/tini -- /usr/lo"   11 hours ago   Up 11 hours   0.0.0.0:8080->8080/tcp,...   jenkins-hyper-demo  23.236.114.227

Of course, the Jenkins server don't have to be hosted on Hyper_.
The key is to install Jenkins and hyper cli together.
For example, you can run the Jenkins server with docker on your own host. Here is an example run jenkins server with docker

$ mkdir -p $(pwd)/jenkins-data
$ ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxx
$ SECRET_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxxx
$ docker run -d --name jenkins-hyper-demo \
  -v $(pwd)/jenkins-data:/var/jenkins_home \
  -p 8080:8080 \
  -p 50000:50000 \
  -e ACCESS_KEY=$ACCESS_KEY \
  -e SECRET_KEY=$SECRET_KEY \
  hyperhq/jenkins-hypercli:2.10
//get tenantid to test the connection to Hyper_
$ docker exec -it jenkins-hyper-demo hyper info | grep ID

But if you want to receive GitHub webhook to trigger Jenkins to start build, you need a public ip for this container.

Step 2. Config Jenkins

open Jenkins web ui: http://23.236.114.227:8080

2.1 initialize Jenkins(first time)

screenshot

Unlock Jenkins(with unlock password)
  -> Customize Jenkins(install suggested plugins)
    -> Create First Admin User
      -> Jenkins is ready!

2.2 add github server

screenshot

Manage Jenkins -> Configure System -> GitHub
  -> Add Github Server
    -> Add Credentials (kind="Secret text", Secret=<github personal access token>)
      -> Test connection
        -> Manage hooks(checked)

2.3 create new job for build example project

screenshot

Require:
- AWS S3 credential
- S3 Bucket

New Item -> Freestyle project, input item name: "demo"
  -> Source Code Management 
       Repository URL="https://github.com/hyperhq/jenkins-hyper-example-app",
       Branch="*/master" 
  -> Build Triggers
       Build when a change is pushed to GitHub(checked)
  -> Build
    -> Execute shell
        Command:

The following is the content of Command:

### prepare
# remove old container
(hyper ps -a | grep jenkins-hyper-example-builder) && hyper rm -f jenkins-hyper-example-builder
# pull new image
hyper pull hyperhq/jenkins-hyper-example-builder

### start new container (set AWS S3 Credential and S3 Bucket here)
AWS_ACCESS_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx
AWS_SECRET_KEY=xxxxxxxxxxxxxxxxxxxxxxxxxx
AWS_S3_BUCKET=jenkins-hyper-example-app
hyper run -d --name jenkins-hyper-example-builder \
 --size=l \
 -e AWS_ACCESS_KEY=$AWS_ACCESS_KEY \
 -e AWS_SECRET_KEY=$AWS_SECRET_KEY \
 -e AWS_S3_BUCKET=$AWS_S3_BUCKET \
 hyperhq/jenkins-hyper-example-builder

### start deploy in container ( unit-test -> make -> integration-test -> upload )
# clone source from github
hyper exec -i jenkins-hyper-example-builder /step.sh clone
# start deploy
hyper exec -i jenkins-hyper-example-builder /go/src/github.com/hyperhq/jenkins-hyper-example-app/auto.sh

### clear container
hyper stop jenkins-hyper-example-builder
hyper rm jenkins-hyper-example-builder

Step 3. Trigger Build

screenshot

//trigger by manual
click "Build Now" link.

//trigger by github webhook
just push change to GitHub

Step 4. View job result

screenshot

Build History -> Console Output