Full Tutorial: Minishift - ocd-scm/ocd-meta GitHub Wiki

These instructions show how to set up an engineering (build) environment, a staging (test) environment, and a live environment on minishift that runs a reactjs app from realworld.io. There is another set of instructions that show how to run the same demo on openshift online pro using github to host all the sample configuration.

Here is a sequence diagram of the components that we will install into each project that represents a logical environment:

Referring to the above diagram:

• We will install Gitea an opensource git server that works like github within minishift. It will fire webhook events to drive installing the configuration from git push event and start app builds from git release (tag) events.
• We will install one instance of ocd-environment-webhook per logical environment. This is a small adnanh/webhook go app that checks out our config rep and runs helmfile to install and upgrade our applications.
• We will run on an instance of Tiller the Helm server. This will install all the OCD releases defined in the helmfile.yaml file in our environment git repo. These are git webhook handlers, builds that create container images based on git release (tag) events, deployments to run our tagged containers, as well as secrets and config maps that configure our apps.

You need to:

1. Install Minishift and get Helm and Tiller running. See these instructions Helm Tiller on Minishift.
2. Install Gitea on Minishift. See these instructions Gitea On Minishift

That wiki page also explains how to setup a personal access token which is some uuid like "f64960a3a63f5b6ac17916c9be2dad8dc76c7131" which can be used to push changes without a username and password.

After you have installed Gitea using the instruction at the like above create an empty repo realworld-react-redux then use the following to clone and demo code and push it into Gitea:

git clone https://github.com/simbo1905/react-redux-realworld-example-app realworld-react-redux
cd realworld-react-redux
# this should print the gitea url. If it doesn't set it manually: 
oc project gitea
GITEA_URL=$(oc get routes | awk '$1~/gitea/{print $2}')
echo $GITEA_URL
# set this to an access token created in Gitea to avoid having to use a username and password
ACCESS_TOKEN=f64960a3a63f5b6ac17916c9be2dad8dc76c7131
# set this to your username in gitea needed to compute the url to your repo below 
USER_NAME=simbo1905
# add the gitea repo as a remote
git remote add minishift http://$ACCESS_TOKEN@$GITEA_URL/$USER_NAME/realworld-react-redux.git
# push the code into Gitea
git push minishift master

Now we need to create a second repo for the config that will deploy our app. Create an empty Gitea repo on minishift called env-eng and push the demo config into it:

cd ..
git clone https://github.com/ocd-scm/ocd-demo-env-build
cd ocd-demo-env-build
# this should print the gitea url. If it doesn't set it manually: 
oc project gitea
GITEA_URL=$(oc get routes | awk '$1~/gitea/{print $2}')
echo $GITEA_URL
# set this to an access token created in Gitea to avoid having to use a username and password
ACCESS_TOKEN=f64960a3a63f5b6ac17916c9be2dad8dc76c7131
# set this to your username in gitea needed to compute the url to your repo below 
USER_NAME=simbo1905
# add the gitea repo as a remote
git remote add minishift http://$ACCESS_TOKEN@$GITEA_URL/$USER_NAME/ocd-demo-env-build.git
# push the code into Gitea
git push minishift master

Edit the config file ./realworld-react-redux/envvars in your env-eng to point to your gitea copy of the realworld repo. This is so that you build can your own version of the demo code from your own git release tags using your own gitea release webhooks events. Now commit this change to the config repo and push it up to gitea.

Okay, now we have the prerequisite components we can set up an engineering (build) environment. Set an env var to be your project name:

export PROJECT=your-eng

Create the oc project and load the passphrase for the GPG private key that is in the demo configuration:

oc new-project $PROJECT
oc create -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: openshift-passphrase
stringData:
  passphrase: 026Kwzeu+8OyC6VyfWJM
EOF

Run the ocd-environment-webhook wizard.sh to install the ocd-environment webhook into minishift giving it the gitea repo URL of the demo config (including your personal access token). You will be asked for the name of the build image namespace which is the current project. Use the defaults for the rest:

pushd /tmp && oc project $PROJECT && curl -L https://github.com/ocd-scm/ocd-environment-webhook/archive/v1.0.1.tar.gz | tar zxf - \
&& cd ocd-environment-webhook-1.0.1 \
&& ./wizard.sh && popd

Here is a transcript of running on my local minishift:

○ → ./wizard.sh 
OCD_PROJECT=your-eng
OCD_SERVER=192.168.99.100:8443
The git repo url? http://8a0b82563b5c93f3b1137dd21be60b6c8c85569e@gitea-gitea.192.168.99.100.nip.io/simbo1905/env-eng.git
The project where the images are built and promoted from? your-eng
Repo name? (default: simbo1905/env-eng): 
Branch ref? (default: refs/heads/master): 
Chart instance prefix? (default: your-eng): 
Use --insecure-no-tls-verify? (default: false): 
Chatbot base64 encoded notify script? 
Value of --oc-server: 192.168.99.100:8443

Check that it has successfully deployed the release which would look something like:

$ helm list | awk -v PROJECT=$PROJECT '$NF==PROJECT{print $0}'
your-eng-ocd-webhook    2               Sun Jan 20 01:50:22 2019        DEPLOYED        ocd-environment-webhook-1.0.1                   your-eng

Next setup a webhook on Gitea for the env-eng to hit the oc-environment-webhook supplying the secret. We can generate the host/port of the webhook with an oc lookup:

oc get route ocd-environment | awk 'NR>1{print "http://" $2 "/hooks/ocd-environment-webhook"}'

We can set up the gitea webhook of type application-json to push all events to the URL of the ocd-environment webhook with the secret shown by:

oc describe dc ocd-environment-webhook | grep WEBHOOK_SECRET | awk '{print $2}'

Here is what the webhook looks like on my gitea:

You fire the webhook by either using the gitea web ui to edit the readme.md or by pushing to the gitea repo. You should see in the logs of the webhook deployment that it run helmfile to install the webhook that will build our app from a git release event. This will appear on the overview:

We can also list in helm the releases that it has installed:

$ helm list | awk -v PROJECT=$PROJECT '$NF==PROJECT{print $0}'
your-eng-ocd-webhook                    1               Sun Jan 20 03:05:00 2019        DEPLOYED        ocd-environment-webhook-1.0.0                   your-eng
your-eng-realworld-react-redux          1               Sun Jan 20 03:05:31 2019        DEPLOYED        ocd-builder-1.0.0                               your-eng
your-eng-realworld-react-redux-webhook  1               Sun Jan 20 03:05:31 2019        DEPLOYED        ocd-builder-webhook-1.0.0                       your-eng

Now we can create a webhook on the gitea demo code repo of type 'application/json' for events 'releases' with the URL shown by:

oc get route webhook-realworld | awk 'NR>1{print "http://" $2 "/hooks/ocd-builder"}'

and with the secret shown by:

oc describe dc webhook-realworld | grep WEBHOOK_SECRET | awk '{print $2}'

Now on the gitea web ui go to 'release' create a new release of the demo code starting with 'v' such as 'v0.0.1'.

Clone the staging config repo at ocd-scm/ocd-demo-env-staging. Create an empty Gitea repo on minishift called env-staging and push the demo config into it:

git remote add minishift http://<personal-access-token>@<gitea-url.<your-ip>.nip.io/your/env-staging
git push minishift master

Set up the openshift project and load the GPG passphrase to decrypt config secrets:

export PROJECT=your-staging
oc new-project $PROJECT
oc create -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: openshift-passphrase
stringData:
  passphrase: 026Kwzeu+8OyC6VyfWJM
EOF

Obviously, you would create your own passphrase protected GPG key but that one is what is used in the demo git repo you uploaded into Gitea.

Run the ocd-environment-webhook wizard.sh to install the ocd-environment webhook into minishift giving it the gitea repo URL of the staging config (including your personal access token). Set the build namespace to be your-eng and use the defaults for the rest.

Now print out the webhook URL and secret to configure a gitea webhook to fire config changes into our project:

# print the URL
oc get route ocd-environment | awk 'NR>1{print "http://" $2 "/hooks/ocd-environment-webhook"}'
# print the webhook secret
oc describe dc ocd-environment-webhook | grep WEBHOOK_SECRET | awk '{print $2}'

In Gitea setup a new webhook on the your-staging of type json repo using the URL and secret printed by those commands. Hit the 'Test Webhook' button and watch the logs of the webhook to see it run. Check what it installs:

$ helm list --namespace=your-staging
NAME                            REVISION        UPDATED                         STATUS          CHART                           APP VERSION     NAMESPACE
your-eng-ocd-webhook            1               Sun Jan 27 07:35:48 2019        DEPLOYED        ocd-environment-webhook-1.0.0                   your-eng
your-eng-realworld              1               Sun Jan 27 07:36:19 2019        DEPLOYED        ocd-builder-1.0.0                               your-eng
your-eng-realworld-webhook      1               Sun Jan 27 07:36:19 2019        DEPLOYED        ocd-builder-webhook-1.0.0                       your-eng

We should also see the new app deploy and the url to open up the running app is shown by:

$ oc get route realworld-react-redux | awk '{print $2}'
HOST/PORT
realworld-react-redux-your-staging.192.168.99.100.nip.io

This is identical to the staging setup. We recommend a repo per environment as git repo permission are on a repo level and you might want to restrict who is writing to the production git repo when it is automatically deployed. We should also note that config and secrets don't flow between environments so branches are not an entirely natural solution.

oc new-project your-live

Clone the staging config repo at ocd-scm/ocd-demo-env-live. Create an empty Gitea repo on minishift called env-live and push the demo config into it:

git remote add minishift http://<personal-access-token>@<gitea-url.<your-ip>.nip.io/your/env-staging
git push minishift master

Set up the openshift project and load the GPG passphrase to decrypt config secrets:

export PROJECT=your-live
oc new-project $PROJECT
oc create -f - <<EOF
apiVersion: v1
kind: Secret
metadata:
  name: openshift-passphrase
stringData:
  passphrase: 026Kwzeu+8OyC6VyfWJM
EOF

Obviously, you would create your own passphrase protected GPG key but that one is what is used in the demo git repo you uploaded into Gitea.

Run the ocd-environment-webhook wizard.sh to install the ocd-environment webhook into minishift giving it the gitea repo URL of the live config (including your personal access token). You will also need to set the build namespace to be your-eng and use the defaults for the rest.

Now print out the webhook URL and secret to configure a gitea webhook to fire config changes into our project:

# print the URL
oc get route ocd-environment | awk 'NR>1{print "http://" $2 "/hooks/ocd-environment-webhook"}'
# print the webhook secret
oc describe dc ocd-environment-webhook | grep WEBHOOK_SECRET | awk '{print $2}'

In Gitea setup a new webhook on the your-live of type json repo using the URL and secret printed by those commands. Hit the 'Test Webhook' button and watch the logs of the webhook to see it run. Check what it installs:

$ helm list --namespace=your-live
NAME                            REVISION        UPDATED                         STATUS          CHART                           APP VERSION     NAMESPACE
your-eng-ocd-webhook            1               Sun Jan 27 07:35:48 2019        DEPLOYED        ocd-environment-webhook-1.0.0                   your-eng
your-eng-realworld              1               Sun Jan 27 07:36:19 2019        DEPLOYED        ocd-builder-1.0.0                               your-eng
your-eng-realworld-webhook      1               Sun Jan 27 07:36:19 2019        DEPLOYED        ocd-builder-webhook-1.0.0                       your-eng

We should also see the new app deploy and the url to open up the running app is shown by:

$ oc get route realworld-react-redux | awk '{print $2}'
HOST/PORT
realworld-react-redux-your-staging.192.168.99.100.nip.io

wizard.sh errors saying Error from server (Forbidden): .. User "developer" can happen after restarting minishift as the permission don't seem to flush to etcd properly on restart. You need to go back to the minishift setup instruction and rerun the commands to permission the developer user.

Webhook log forbidden permission denied errors saying no RBAC and use x can y z. If you look at oc policy who-can y z it says user x can do that. Doing minishift stop then starting it up again seems to put things back into a consistent state.

See the Example Environment Layout for an overview of how to run a more sophisticated set of apps with shared secrets and automate restarts of apps when secrets have been updated.