Docker, Docker Compose

RSpec, Capybara, Selenium

Terraform

Jenkins

Docker is a platform for running applications and their dependencies in isolated environments called containers, on nearly any operating system. Homebrew (brew.sh)for installing applications on Mac(Linux) Aptitude, YUM, and YaST are package managers that let you install anything on the Mac

$ /bin/bash -c "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/HEAD/install.sh)"

$ brew install cask docker - cask(third-part repository) installs Docker, Docker CLI, Docker Compose

$ docker run hello-world

The Dockerfile is a manifest that describes the image that the container will use when we run it. All of the configuration dependencies and environment dependencies and everything that the application needs will be expressed within the Dockerfile.
Docker will do a few things.

1. The Docker reads and parses the Dockerfile.
2. The Docker fetches the parent image that this image is going to use. If there is no parent image that you're going to use, you would start from scratch.
3. The Docker runs any commands, within the Dockerfile that is on top of that image, lastly if defined you can set a process that runs whenever a container from that image is spun up.

The 'alpine' is the smallest version of the image

FROM nginx:alpine                                   #parent image:tag

MAINTAINER Illya Korotun <[email protected]>  #author
COPY website /website                               #copy context into the image host directory to the container directory
COPY nginx.conf /etc/nginx/nginx.cong               #copy the config file to the container

EXPLOSE 80 #we want to map one(80) of the ports on my Mac to the port on the container.

$ docker build --tag website . # create an image with tag/name "website" from "." (current directory)

$ docker run --publish 80:80 website # publish on ports HOST:Container

Docker compose is a simple and lightweight platform for running multiple container applications in a single stack and creating a network.

version: '3.7'   # docker compose version
services:
  website:       # service name usually equals the container name
    build:       # look for Dockerfile and build an image
      context: . # Dockerfile address "."-current directory
    ports:       # container ports HOST:Container
      - 80:80    # Dockerfile address "."-current directory

$ docker-compose run website

$ docker-compose up

$ docker-compose ps

$ docker ps

$ docker-compose down

• RSpec is a Ruby-based testing framework. RSpec is looking for a folder called Spes and _spec.rb files (ex. page_spec.rb).
• Capybara is a tool that lets you use a web driver to create a browser and interact with the website. It gets the HTML of a website, run Javascript, and tests things like a button looking like it should look.
• The web driver, which is what spins up the browser and runs tests against it.
• Rack Test is a very simple web browser that just renders HTML and tests it, but it can't do more complicated things like running Javascript or computing CSS.
• Selenium is a web driver that spins up a real web browser and tests against whatever the web browser sees, which is nice because our website does have computed CSS, and spinning up a real web browser and getting the results of that computed CSS.

$ mkdir spec
$ mkdir spec/unit

$ touch spec/unit/page_spec.rb

# spec/unit/page_spec.rb
require 'capybara'
require 'capybara/dsl'

describe "Example page render unit tests" do
  it "Should show the Explore California logo" do
  end
end

version: "3.7"
services:
  website:
    build:
      context: .
    ports:
      - 80:80
  unit-test:                       # new service
    volumes:                       # mount the existing volume to the container
      - "$PWD:/app"                #current directory to /app
    build:                         # build a new image
      context: .                   #current context
      dockerfile: rspec.dockerfile #spesific dockerfile
    command:
      - --pattern                  # entrypoint option
      - /app/spec/unit/*_spec.rb        # entrypoint option test target
    

$ touch rspec.dockerfile

# create from base ruby image
FROM ruby:alpine 
MAINTAINER Illy Korotun <[email protected]>

# apk is pacage from alpine
# add package ruby-nokogiri for parsing 
RUN apk add build-base ruby-nokogiri

# install in the container from the ruby gem library rubygems.org
RUN gem install rspec capybara selenium-webdriver

#Which process should start when the container is starting. Use double equate 
ENTRYPOINT ["rspec"]

Two way say container how to run CMD and ENTRYPOINT
Every container start with the default command #/bin/sh -c
CMD ["rspec"] # is mean #/bin/sh -c "rspec"

ENTRYPOINT is running the command where it placed
ENTRYPOINT ['option1', 'option2'] # is mean docker run this_image --option1 --option2
ENTRYPOINT ['rspec'] # is mean docker run this_image rspec

$ docker-compose up -d website

$ docker-compose run --rm unit-test
The dot '.' should be shown as a result of the successful test

$ docker-compose config

$ docker-compose up -d website

# spec/unit/page_spec.rb
require 'capybara'
require 'capybara/dsl'
require 'selenium-webdriver'

include Capybara::DSL
Capybara.app_host = "http://website" # Using Selenium; connect over network
Capybara.run_server = false # Disable Rack since we are using Selenium.
Capybara.register_driver :selenium do |app|
  Capybara::Selenium::Driver.new(
    app,
    browser: :remote,
    url: "http://#{ENV['SELENIUM_HOST']}:#{ENV['SELENIUM_PORT']}/wd/hub",
    capabilities: Selenium::WebDriver::Remote::Capabilities.chrome( # desired_capabilities was replaced by capabilities in new Capybara 4
      "chromeOptions" => {
        "args" => ['--no-default-browser-check']
      }
    )
  )
end
Capybara.default_driver = :selenium

describe "Example page render unit tests" do
  it "Shows the Explore California logo" do
    visit('/')
    expect(page.has_selector? '.logo').to be true
  end
end

version: "3.7"
services:
  selenium:
    image: selenium/standalone-chrome-debug # debug version include VNC
    ports:
      - 4444:4444
      - 5901:5900 # VNC ports. The 5900 is the default VNC port
  unit-test: # new service
    build: # build a new image
      dockerfile: rspec.dockerfile #spesific dockerfile
      context: . #current context
    volumes: # mount the existing volume to the container
      - $PWD:/app #current directory to /app
    command:
      - --pattern # entrypoint option
      - /app/spec/unit/*_spec.rb # entrypoint option test target
  website:
    build:
      context: .
    ports:
      - 80:80

$ docker-compose up -d website selenium
$ docker-compose run --rm unit-test

localhost:5901
password: secret

$ docker-compose run --rm unit-test

Terraform is an open-source infrastructure-as-configuration software tool created by HashiCorp. Users define and provide data center infrastructure using a declarative configuration language known as HashiCorp Configuration Language, or optionally JSON. the file is saved in .terraform/terraform.tfstate Terraform backend is usually stored in AWS S3 Terraform plugins which created by providers CA certificate add CA-certificate in docker file

# create from Linux alpine
FROM alpine
MAINTAINER Illya Korotun <[email protected]>

#download Terraform from https://developer.hashicorp.com/terraform/downloads

RUN wget -O /tmp/terraform.zip https://releases.hashicorp.com/terraform/0.12.12/terraform_0.12.12_linux_amd64.zip
RUN unzip /tmp/terraform.zip -d /

# add CA-certificate
RUN apk add --no-cache ca-certificates curl

# nobody user has privileges to execute
USER nobody 

# check installed terraform in the container
ENTRYPOINT [ "/terraform" ]

$ docker build -t terraform . -f terraform.Dockerfile

$ docker run --rm --interactive --tty --entrypoint sh terraform
~$ /terraform #check installed terraform in the container
~$ exit

docker build --tag terraform --file terraform.dockerfile .

docker run --rm terraform -version

...
service:
  terraform:
    build:
      dockerfile: terraform.dockerfile
      context: .
    environment:
      AWS_ACCESS_KEY_ID: "${AWS_ACCESS_KEY_ID}"
      AWS_SECRET_ACCESS_KEY: "${AWS_SECRET_ACCESS_KEY}"
      AWS_REGION: "${AWS_REGION}"
    volumes:
      - $PWD:/app
    working_dir: /app
...

$ docker-compose run --rm terraform

AWS - Amazon Web Service is a cloud provider of resources for Amazon data center.
Elastic Cloud Compute or EC2 allows the creation of virtual machines.
Amazon S3 is a simple storage service.
The project will be copied into the bucket of the S3.

data "aws_iam_policy_document" "bucket_policy" {
  statement {
    sid = "PublicReadGetObject"
    effect = "Allow"
    actions = [ "s3:GetObject" ]
    principals {
      type = "*"
      identifiers = [ "*" ]
    }
    resources = [ "arn:aws:s3:::explorecalifornia.org/*" ]
  }
}

/* We can access properties from data sources using this format:
   ${data.<data_source_type>.<data_source_name>.<property>.

   In this case, we need the JSON document, which the documentation
   says can be accessed from the .json property. */

resource "aws_s3_bucket" "website" {
  bucket = "explorecalifornia.org"  // The name of the bucket. It should be unique. 
  acl    = "public-read"            /* Access control list for the bucket.
                                       Websites need to be publicly-available
                                       to the Internet for website hosting to
                                       work. */
  policy = "${data.aws_iam_policy_document.bucket_policy.json}"
  website {
    index_document = "index.htm"   // The root of the website.
    error_document = "error.htm"   // The page to show when people hit invalid pages.
  }
}

output "website_bucket_url" {
  value = "${aws_s3_bucket.website.website_endpoint}"
}

The terraform state should be removed from the public repository.

docker-compose run --rm terraform init

docker-compose run --rm terraform plan
Enter a value: us-east1 // closest server location

docker-compose run --rm terraform apply

services:
...
  aws:
    image: organs/awscli
    environment:
      AWS_ACCESS_KEY_ID: "${AWS_ACCESS_KEY_ID}"
      AWS_SECRET_ACCESS_KEY: "${AWS_SECRET_ACCESS_KEY}"
      AWS_REGION: "${AWS_REGION}"
      AWS_DEFAULT_REGION: us-west-2
    volumes:
      - $PWD:/app
    working_dir: /app
...

The default entrypoint 'sh' $ docker-compose run --rm aws

$ docker-compose run --rm --entrypoint aws aws ec2 describe-instances

$ docker-compose run -rm terraform output

$ docker-compose run -rm --entrypoint aws aws s3 cp --recursive website/ s3://explorecalifornia.org

$ docker-compose run -rm --entrypoint aws aws s3 ls s3://explorecalifornia.org*

$ docker-compose run -rm --entrypoint aws aws s3 rm s3://explorecalifornia.org --recursive

$ docker-compose run -rm terraform destroy

$ mkdir spec/integration $ cp spec/unit/page_spec.rb spec/integration/

include Capybara::DSL
Capybara.app_host = "http://#{ENV['WEBSITE_URL']}"
...

docker-compose run --rm terraform plan docker-compose run --rm terraform apply

services:
...
  integration-test: # new service
    environment:
      SELENIUM_HOST: selenium
      SELENIUM_PORT: 4444
    volumes: # mount existen volume to container
      - $PWD:/app #current directorin to /app
    build: # build new image
      dockerfile: rspec.dockerfile #spesific dockerfile
      context: . #current context
    command:
      - --pattern # entrypoint option
      - /app/integration/unit/*_spec.rb # entrypoint option test target  
...

$ docker-compose run -rm --entrypoint aws aws s3 cp --recursive website/ s3://explorecalifornia.org $ docker-compose up -d selenium
Start VNC session localhost:5901
$ docker-compose run --rm -e WEBSITE_URL=your_remote_site_url integration-test "." is mean that it is passing test

$ docker-compose run -rm --entrypoint aws aws s3 rm s3://explorecalifornia.org --recursive

$ docker-compose run -rm terraform destroy

$ docker-compose down

# image from hub.docker.com
FROM jenkins/jenkins:alpine
MAINTAINER Illya Korotun <[email protected]>

# Jenkins plugins on [plugins.jenkins.io](https://plugins.jenkins.io/)
COPY plugins.txt /usr/share/jenkins/plugins.txt

# Install Jenkins plugins
# RUN /usr/local/bin/install-plugins.sh < /usr/share/jenkins/plugins.txt
RUN jenkins-plugin-cli -f /usr/share/jenkins/plugins.txt

# Expose Jenkins default port
EXPOSE 8080

workflow-aggregator
seed
git

...
services:
  jenkins:
    build:
      context: .
      dockerfile: jenkins.dockerfile
    environment:
      JAVA_OPTS: "-Dhudson.plugins.git.GitSCM.ALLOW_LOCAL_CHECKOUT=true" 
    volumes:
      - $PWD/jenkins_home:/var/jenkins_home #Jenkins working directory on HOST : Container
      - $PWD:/app #current directory for app HOST:Container change to Git
    ports:
      - 8080:8080 # HOST_port:Container_port
...

$ docker-compose up Jenkins

username: admin password in the log

Jenkins initial setup is required. An admin user has been created and a password generated.

Please use the following password to proceed with the installation:


ccc96995dba047faa4280bc57471d001


This may also be found at: /var/jenkins_home/secrets/initialAdminPassword

or in the file /var/jenkins_home/secrets/initialAdminPassword
Not commit Jenkins home to Git repository Start using Jenkins

declarative Jenkins pipeline (more readable)
scriptive Jenkins pipeline (Grovy)

pipeline {    //
  agent any  // what kind of Jenkins slaves to build this job on
  stages {  // stages define the actual stages that need to run. And these run serially by default, so they run step-by-step. You can configure it to run in parallel. 
    stage('Build our website'){  // part of the CI
      steps {  // Run sh script
        sh "scripts/build.sh" // invoce the shell and run script
      }
    }
    stage('Run unit tests'){  // part of the CI
      steps {  // Run sh script
        sh "scripts/unit_test.sh" // invoce the shell and run script
      }
    }
    // stage('Run integration test'){}  // part of the CI

    stage('Deploy website'){  // part of the CI
      steps {  // Run sh script
        sh "scripts/deploy_website.sh" // invoce the shell and run script
      }
    }
  }
}

echo "I am running: $0"

echo "I am running: $0"

echo "I am running: $0"

$ git init

node_modules
jenkins_home
.terraform
.DS_Store
.idea/*

git checkout -b master // production ready branch

git add .

git commit -m "Initialize repository."

docker-compose up jenkins

username: admin password: in /var/jenkins_home/secrets/initialAdminPassword

Create job
Enter an item name: "explore_california_ci"
Multibranch Pipeline - Ok
Configuration:
General:
Display Name: Explore California CI
Description: This pipeline deploys the Explore California website.
Branch Sources: Add Source: Git: Project Repository: file:///app (because the code was mounted directly in the container) or https://github.com/ILLYAKO/DevOps-foundations.git
Build Configuration
Mode by Jenkins
Script Path Jenkins
Save - Apply