CI CD Jenkins XlDeploy WebContainers - LearningRbcRegistry/Wiki GitHub Wiki

#CI / CD / Jenkins / XL Deploy#
Antoine Bour / Nov 16*

1. Continuous Integration and Continuous Delivery

a. Continuous Integration
b. How to?
c. Benefits

2. Continuous Deployment

3 Continuous Integration with Jenkins

4 Groovy

5 XL Deploy

6 In practice

7 Was and Web containers

8 Links

9 Deeper in details

1. Continuous Integration and Continuous Delivery

a. Continuous Integration
  • Software development practice where team members integrate their work frequently
  • Each integration is verified by an automated build
  • Including tests
  • It reduce integration problems (work is fully automated)
b. How to?
  • Maintain a single source Repository: git
    • Everyone commits to the Mainline every day
  • Automate the build
    • Each commit trigger a build
    • Environment per environment
    • Automated Testing (front and back) is part of the process
    • Fix the bug (identified by a broken build) immediately
  • Keep the build as fast as possible
  • Test in a clone of the production environment
    • Try to have the same infrastructure
    • With the same database
    • With the same level of datas (performance check)
  • Build result should be visible for all
c. Benefits
  • Reduce integration time
  • Reduce bug (Easier to detect, to fix..)
  • Allow frequent deployments
  • Quality metrics

2. Continuous Deployment

  • We are no more talking to deployment in staging… but now let’s deploy to production:
  • Requires collaboration between Development team and support team

3 Continuous Integration with Jenkins

  • Jenkins is a fork from Hudson
  • Jenkins (through plugins) provide the code quality or code coverage metrics
  • Jenkins Pipeline:
    • The project deployment phasis can be contained inside a JenkinsFile
    • Requires the corresponding plugin
    • Whole project build/test/deployment is managed in this file
    • Can be part of git
    • Ability to suspend/resume a job
    • Writing it’s own step is possible.
  • In fact we are allowed to:
    • Manage build for each wished git branches
    • Automate the build through the pipeline thanks to groovy
  • Pipeline:
    • Segmented in differents “stage” (stage build, stage test)
    • Work from a stage is contained in “node” block (They run a Jenkins agent node)
    • A stage can be composed of differents “steps”
    • Parallel work is possible
    • Input are possible (Do you approve this deployment?)

4 Groovy

String handling + overloading String StringTest.groovy
Regular Expressions RegularExpressionTest.groovy
Property access for a list Person.class
Convert a list to an array (and opposite) ArrayToList.groovy
Spreaddot operator: ArrayToList.groovy
Maps: MapTest.groovy
Each, any and every methods: CallMethods.groovy
Loop
Processing files: (to the console) printLoop.groovy
Processing HTTP requests. Using template engines TemplateEngine.groovy
XML
JSON adoption is easy XmlGroovy.groovy
Abstract Syntax Tree Transformation, or in Memory representation of code as data
Using groovy in java
Calling a script Refer to jenkinsfile of a project

5 XL Deploy

  • Deployment automation (from war file to websphere)

    • Security mechanism that manage roles and permissions
    • Plugins that add capabilities

  • Configuration Items:

    • Directories: Roles/permission management
    • Deployable (war/ear/html)
    • Resources (datasource, queue, connection factory)
    • Composites (dependances)
    • Dictionaries (specific environment entries, using placeholder)
    • Container (Websphere, tomcat)
    • Environment (Dev/test/Prod)
    • Deployment/undeployment/rollback…

  • Through Jenkins or maven plugin, a DAR (Deployment archive) file is created. This package can be included in DeployIt.

6 In practice

  • Prerequisite: - Java is installed - Your work is on git, git is installed on your environment.

  • My backend: https://github.com/Assignment2016/RBCITS_Assignment_Bour_2016.git

  • My front end: https://github.com/LearningRbcRegistry/TSAngularAndCo.git

  • Install Tomcat

  • Install Jenkins in tomcat (Jenkins.war to put in tomcat/webapps)

  • Install XlDeploy (which requires Tomcat as a service)

    • Installation of a new service for tomcat
    • Download and install of XlDeploy free trial version version (valid only 1 month)
    • Deploy XLDeploy

  • The backend project was initially developed to start inside spring-boot. Some updates have been applied:

    • Move to war file
    • Manage dependencies (jpa issues)
    • Move from hsqldb to MySql
    • Check the start of the project directly with tomcat

  • Jenkins setup:

    • Install the recommended plugins
    • Install git and githib plugins
    • Install nodejs plugin
    • Install Pipeline plugin
    • Jenkins contains a lot of usefull global variables (pipeline->pipeline syntax) that helps with git.

  • Configure NodeJS (Jenkins-> Configuration)

  • Configure JDK Path
  • Configure Maven Path
  • Configure Shell and Configure XlDeploy:
  • Light Pipeline for the Backend Project:
    • Define a pipeline that retrieve the project in git and build a mvn package:
  • The build result is the following (blue = success):

The build can be automated with the Help from git that can communicate to Jenkins at each commit in git. In the Lab, due to firewall issues, they are polling git each 5 minutes.

  • 2nd test, create an item, just for testing the XL deploy plugin: - No source management for this test (no link to git) - JDK is mapped to our local JDK installation (already configured in Jenkins global configuration) - Configure the XL Deploy post build action:
  • It should be possible to upload the created “dar” file to XLDeploy. This is not working due to restriction accesses on the used computer.
  • A DAR file is generated. We will include it manually in Deploy IT.
  • Light Pipeline for the front end Project:

The final result is the following:

  • But what does this job contains? - First remind that nodeJs is installed.

The project is build in the Jenkins webapp folder. This is not the goal of a pipeline, but through “gulp” we have launched the front end.

In order to be able to deploy the front end project, it has been necessary to resolve all peer dependencies in the npm project configuration.

  • Xl Deploy integration (http://localhost:4516)

    • Create or Import a package (Tab Deployment):
      • Import a package: redirect to our created DAR file
      • Create a package: Redirect to the War file existing in webapps from tomcat
  • Create a receiving infrastructure: In our case a Tomcat that run as a service.
    • Tab Repository, create the following:
      • Rightclic, new, overthere, localhost.
        • The Operating System is WINDOWS in this case.
        • Name it LocalInfra
        • Save
    • Rightclic on LocalInfra, new, tomcat,server.

More details about commands.

  • Now Create an environment (right green panel)
    • Name it “local” and point to the created container “localInfra’
  • Drag and drop your application in the central left panel. Do the same for the wished environment.
  • And execute the deployment:
  • Start your application
  • My Application is up and running:

- Conclusion

  • I was able to retrieve a project from git.
  • To build it, with or without the tests
  • Same for the front end
  • The backend packaging in a Dar is easy through the XLdeploy plugin
    • The XL Deploy plugin is not working in a pipeline, that’s why it has been done through an Item. Another possibility is to use groovy to do the same (through the XL Deploy maven plugin or by forking the plugin code)
  • The configuration of XL Deploy is quite easy for a simple project.
  • In real conditions we should add more capabilities:
    • Working with branches
    • Versioning inclusion
    • Upload in Jenkins capabilities
    • Maven web plugin to package the front end for a web container
    • Managing multiple environment through Jenkins AND XLDeploy
    • Full automated flow.

7 Was and Web containers

8 Links

9 Deeper in details

  • Jenkins config for tags (snapshot)
  • Branching, from Jenkins to XlDeploy
  • Include a placeholder and define a dictionary in XLdeploy
  • Define datasource as parameters
  • …..
⚠️ **GitHub.com Fallback** ⚠️