include ref=HeaderPage

Step 1: Local Development

• create/select your stage (i.e. I use my initials to name it)
• write code (like in the olden days!)
• store your configuration and secrets in JSON files (environment variables generated for you)
• update your databases and schemas (i.e. database deltas)
• write Dockerfiles (with Handlebars expressions to insert configuration data)
• write Kubernetes files (with Handlebars expressions to insert configuration data)

Step 2: Push to Archive

• push code and configuration/secrets to Git (secure repo for configuration/secrets)
• push generated Kubernetes yaml files (Deployments, ConfigMaps, Secrets, Ingress, Services, etc) to secure repository
• push containers to Docker repository
• push database deltas to repository

Step 3: Deploy to Cluster

• Select the version of your system to deploy (defaults to most recent)
• Downloads Kubernetes files and pushes to running cluster
• Use Wovtools CLI to peer into, modify and tinker with running cluster
• Debug a service locally (port forwarding) or on a development pod (Service rerouting)
• Return to step 1 as needed

Step 4: Merge to next Stage

• Change git branch and Kubernetes context to next stage (i.e. 'dev', 'prod', 'security', etc)
• Merge in git repos to this stage (code, configuration, database deltas)
• go to step 1.

Read the Wiki for more information: Wiki Home

Our Facebook plugin connects to Facebook, listening for updates and pushing content to our Woveon system. We run it in our own cluster that we call wov-aws-va-live, in its own project plfb, which we manage in three Kubernetes namespaces of plfb-cw (cw is my private namespace, where 'cw' are my initials), plfb-dev and plfb-prod. Here's how the naming convetions play out in detail:

The Kubernetes context for the production running Facebook plugin is wov-aws-va-live-plfb-prod. This means the cluster name is wov-aws-va-live, implying this cluster is the main Woveon cluster (sys=wov), on Amazon Web Services (provider=aws) in AWS's region us-east-1 (regioncode=va) (i.e. that is in Virginia (va)), on our main cluster (flavor=live) (We could make another cluster with flavor 'top' or 'brown' if we wanted, to experiment with cluster deployment, but chose 'live' for now). The plugin is one project, plfb (short for plugin Facebook), and follows our internal naming convetion of starting plugins with 'pl' followed by the plugin's short code, 'fb' for Facebook. There is one github repo for this project, and each namespace is its own branch (cw, dev, prod). Inside the project, there are three microservices (i.e. three Kubernetes deployments) that append their microservice code to the project to generate its name (wl - plfbwl - the Facebook plugin WoveonListener, rl - plfbrl - the Facebook plugin RemoteListener, etc.). Each microservice will have its own implementation, either a Helm chart or its own Deployment with its own container (plfbwl, plfbrl, etc.). Our cluster has two nodes, which we use kube-aws to manage and at the moment, because we have not set up roles, one user (admin).

1. Project Creation - Run wov-init to transform your existing project into a WovTools project. This will make changes to your git setup, and create a wovtools directory.

2. Create K8s Context - Make sure you have a context in Kubernetes for your namespace ({PROJECT}_{STAGE}).

3. Set Stage - By default, wov-init will exist in your user's stage, but wov-stage <stage> sets the stage for development. So, wov-stage dev or wov-stage prod are two different stages. However, I use my initials for stages, creating my own Kubernetes namespace and git branch to work in. So, wov-stage cw is what I start in.

4. Secrets - You need to start moving configuration information into json files in wovtools/secrets. These files are merged together according to the order in wovtools/secrets/config.json. Also, secrets need to be managed in a safe way. They are stored in a git repository, so we can version the secrets, but you need to ensure the repo is hosted in a secure/encryped location. NOTE: GitHub is not encrypted by default. While your commands will differ, you will use commands similiar to this to commit your secrets to Git:

git remote add origin *URL*    # set origin to remote URL
  # ex. for AWS CodeCommit: git remote add origin https://git-codecommit.us-east-1.amazonaws.com/v1/repos/reponame  
git remote -v                  # test connection
git push -u origin master      # send data to remote master

NOTE: on AWS, you will need to configure your AWSCodeCommit access as well as a git key which git will ask for.

5. Env: variables - Using environment variables is the safe way to run containers and Kubernetes, as you add them at runtime and do not store them in the repo. Create .wov files in 'wovtools/conf' which are compiled for use in bash shells, Makefiles, Kubernetes ConfigMaps and Secrets, etc. Use .ck8s.wov for non-secret data and .sk8s.wov for data that needs to remain encryped (passwords, keys, etc). See wov File Format below.

6. Env: Kubernetes Yaml - Move your Kubernetes yaml files into the wovtools/k8s directory and append the wov extension. Then modify the files to use the secrets you compile from the Env files. NOTE: follows the wov File Format as shown below.

7. Env: Build - Run the wov-build -s command to compile the secrets and then wov-build -e to compile the Env's wov files with this secret. These files are only built as needed (dependencies on timestamps secret diffs) so wov-build -v can print which files are skipped. Also, wov-build -e includes the -s step so just wov-build -e will build the Env in one step.

8. Services - Write scripts that emulate the services your code will interact with in the cluster. This includes databases or other microservices. Write scripts in wovtools/services as SERVICE.sh, SERVICE-test.sh, SERVICE-kill.sh, which correspond to wov-service switches of -r, -t and -k.

9. Cmds - Create commands in the cmds directory that are run with wov-cmd. This are things kile url checks or database access.

10. Develop - Build your code. Whatever you do, create that code. To use environment variables, use the wov-env script to set them, and then set them before your command is run.

# To run service myprojectservice1 with environment vars created with wov-build on file 'wovtools/conf/env.ck8s.wov':
env `wov-env --cmd env.ck8s` npm run myprojectservice1

# To bring the variables locally (not env, so won't be passed to scripts
eval $(wov-env --env env.ck8s)
export WOV_plugin_desc             # now it is global so can be used in a called script


# To use in Makefiles, dump to a file and include 
$(shell wov-env --env env.ck8s > ./.wovtools.mk)
include ./.wovtools.mk

# To get a single WovTools variable (ex. 2):
wov-env --var WOV_PVER   # returns 'WOV_PVER=2'

# To get all WovTools variables:
wov-env -e

1. Git - wov-pushcode-check - Make sure your git archive, and also your wovtools/secret git archive, are checked in and pushed.

2. Containers - Build the containers with the wov-push-containers. This involves creating scripts to create the content that goes into the container. For each container, created a script.sh file in wovtools/containers for that container. Variables passed to the script are CONTAINER (name of the container), SRCDIR (the root of the project) and DESTDIR (the directory the files are dumped). Make sure to include a Dockerfile as well. When your recipes are created, run the wov-push-containers command. To list containers this will build, run wov-push-containers -l.

3. k8s files - Push the env files to archive with wov-push-env.

NOTE: wov-push will do this entire process...

1. Deploy - Run wov-deploy to run the latest project. This pulls env files and runs them. These files in turn include the Kubernetes deployments, which pull the containers. To run older versions, do a wov-deploy --pver X --sver Y to run different versions. Note that only valid combinations of project and secret versions are allowable.