WovTools Configuration - woveon/wovtools GitHub Wiki

< Home

Woveon Logo

As much as possible, the configuration is pushed into the existing tools and directories you already use. When this is not possible, or the data needs to be synced across tools, WovTools manages this and tries to be as simple as possible... but no simpler. Orchestration is complex.

  • WovTools Context is where you are working and deploying. It is an agreement between Git branches, Kubernetes contexts and WovTools Origin. see Naming Conventions

  • WovTools has a Global Configuration directory in your home directory to store defaults and configurations for your WovTools Master and Team Projects.

  • WovTools has a configuration directory 'wovtools' in each Team Project to store both configuration and cached builds.

  • WovTools has Local Archives that stores intermediate builds, before being pushed to Remote Archives. This is by default in the Global Configuration directory.

  • WovTools Vascular Configuration is what stores raw project data and makes it usable and relevant. It is stored in JSON files, the Secrets Files, and uses these to build Context Configurations for WovTools scripts and builds. It is versioned and thus deployed with Docker containers and Kubernetes files.

This takes a bit to wrap your head around. You are not programming the same way anymore. Remember:

  • Your clusters are cattle, not pets. Don't get attached.
  • You write code, engineered to be configured by environment variables provided by scripts or Kubernetes, that is built by a Container Recipe, into a container deployed by Kubernetes, to a cluster defined by your WovTools context.
  • It's easy to work on your own machine but, you don't know if it works until it's in a system and conversely, you don't know if the system works once your code is in it.

WovTools Context with Context Configurations

The WovTools Context is a string that represents where you "are" in your own system. You can have multiple clusters and multiple namespaces, even working and developing locally on your machine yet connected to running services in a cluster. The configuration for executing code in that WovTools Context comes from a Context Configuration built on the fly.

WovTools Context

The WovTools Context string is used by Kubernetes, Git and WovTools to sync your environments.

{context} => [{origin}:]{cluster}-{namespace}
{namespace} => {project}-{stage}

Kubernetes does a good job providing the clusters in its own Kubernetes context. It also separates developers, projects and stages of development with Kubernetes Namespaces. Git uses branches to separate developers and stages of development, and separate repositories for projects and no concept of clusters since Git isn't about execution. So, a WovTools context is defined by a cluster string and a namespace string, but with the namespace string broken down by codes for project and stage. Stages means "of development" (ex. dev, prod, etc) as well as individual developer codes.

Context Configurations

It is standard practice to keep any configuration information separate from your code. However, with WovTools, you can have multiple clusters, repositories, stages of development, etc. Kubernetes needs this information when deploying, as it feeds it to your containers. However, you local environment also needs this information for running scripts to build a new cluster with Kops, check on a running Kubernetes cluster, perform database operations, etc. So, Context Configurations are built on the fly for your WovTools Context (built from your Master Project's Secrets Local Archive) and used in scripts and even building deployable versions of your system.

Together

By syncing naming conventions between Kubernetes and Git, the Git repositories and branches pair to Kubernetes namespaces and Git remains agnostic of the cluster since Context Configurations build the Kubernetes files. Finally, we use origin as an optional portion to a context, since Kubernetes has no concept of not being in a cluster.

WovTools Global Configuration

~/.wovtools/config

The ~/.wovtools/config file impacts how all WovTools projects are managed. It is created by wov-init.

{
  "me" : "...",
  "archives": {
    "coderepo" : "BASE GIT REPO SERVER",
    "container" : "DOCKER CONTAINER REPOSITORY LINK",
    "k8s" : "FILE SERVER FOR k8s files"
  },
  "projects" : {
    "MY PROJECT CODE" : { "dir" : "/path/to/my/wovtools/project" }, ...
  }
}
  • me - This defines your stage code used in WovTools. So, of your project's code is 'abc' and your code is 'def', then your namespace in project 'abc' is 'abc-def'.
  • archives - Default values for the assumed: Code Repository base 'coderepo', Docker Container Repository 'container' and fileserver for Kubernetes files 'k8s'.

'project' MYPROJECTCODE Entries

This provides global configuration information and awareness about your WovTools projects. A project entry is used by wov-cd to quickly change directories (ex. wov-cd abc would take you to '/path/to/abc'). A project entry is also used by wov-checkout to build a WovTools project from all the sources of data and repositories, and hook into local archives.

"PROJECT" : {
  "dir": "/path/to/project/root",
  "repo": "Git repository name ex. MASTERPROJECT_PROJECT",
  "reposerver": "Git repository",
  "sub": [
    # sub directories used
    {
      "atype": "se|db|ds",
      "dir": "/path/to/directory/outside/of/main/project",
      "repo": "Git repository name",
      "repobranch": "Git branch, defaults to master",
      "reposerver": "Git repository"
    },
    ...
  ]
}

Project Environment

WovTools expects systems to have multiple Team Projects (separate Git repos), with one or more microservices in each, which have ownership by one or more developers. Orchestrated systems are too complex to consider in one single project, so:

  • Master Project is the name of the entire group of Team Projects creating an orchestrated system
  • Team Projects can have a single or many microservices (single/many are used below)

The naming of Master and Team projects follows a naming convention so each project is developed and checked out in a common fashion (the Git repository name is the same as the project directory extension):

  • MASTERPROJECT/TEAMPROJECT - for projects with multiple microservices
  • MASTERPROJECT/TEAMPROJECT_MICROSERVICENAME - for projects with one microservice

Project Local Archives

Local Archives store data for a Master/Team Project under development in a Git repo. Local archives are not part of a Team Project repository as they are managed across developers and projects so are linked from the TEAMPROJECT/wovtools directory to an external directory. Local Archives are their own Git repository. There are three types of local archives:

Secret Archive

Secrets (JSON files for building configuration sets) are managed for a specific developer for the entire master project so each dev has their own to avoid users getting unnecessary secrets/access/passwords/etc.

  • is one per developer, per master project
  • linked from wovtools/secrets
  • local directory and repository name: MASTERPROJECT_sea_USERCODE

DataBase Archive

The schemas of WovDatabases are versioned and also link to snapshots. They are managed per project so all devs of that project operate on the same schema.

  • is one per Team Project.
  • linked from from wovtools/db/archive
  • local directory and repository name: MASTERPROJECT_TEAMPROJECT_dba

DataSet Archive

Datasets, as templated files, are stored with the Team Project and the DataSet Archive is a shared repo of json values that provide a set of common values, which is designed to make cross-microservice and cross-project testing consistent.

  • is one per Master Project
  • linked from wovtools/ds/const
  • local directory and repository name: MASTERPROJECT_dsa

Project Config

Project configuration is a mix of wovtools/config.json and wovtools/myconfig.json. The two files are designed so that config.json is fairly static for the project. Your own personal changes can be placed into myconfig.json and kept out of your git repo.

wovtools/config.json

{
  "ver": 3,
  "project": {
    "type": "",
    "project": "mp",
    "masterproject" : "CODE OF MASTER PROJECT",
    "title": "My Project",
    "description": "My microservice WovTools project."
  },
  "archives": {
    "k8s": "...",
    "container": "..."
  },
  "nodeploy": [],
  "secrets": {
    "dev"  : {...},
    "prod" : {...}
  },
  "originmods" : {
    "here"     : {...},
    "internal" : {...},
    "external" : {...}
  }
}

wovtools/myconfig.json

{ 
  "ver-local"    : 3, 
  "curorigin"    : "here",
  "secrets"      : {...},
  "originmods"  : {...}
}

Options Definitions

  • ver/ver-local - show the format version of the file. As the file formats change, so will this number.
  • project - information about this project
    • type - project types can append this text to the microservice name. Leave blank for now. ex. I use 'pl' to append 'pl' to microservices of mine that are plugins.
    • project - the code for the project. This is used to name the microservice(s) in this project. It should also match to the code in ~/.wovtools.
    • title - a formal title for the project.
    • description - a bit about the project.
  • archive -
    • k8s - a file store where files are uploaded to version your project. For now, uses S3 only. ex. s3://my.awsproject.com/archive.
    • container - the Docker Server
  • nodeploy - a list of microservices NOT to deploy. Ignore for now. WovTools stores recipes for creating microservice containers and when it deploys, it deploys all, minus any named here.
  • secrets - a hash of stages for which to deploy secrets for, with a list of files to merge in order. WovTools assumes 'dev' and 'prod'. This is a list of files in a directory linked to from wovtools/secrets that are merged into a single JSON object and out of this object, all configuration is generated. see Secrets and Configuration
  • originmods - modifications to apply to a cluster's generated secrets, to support deployment to a new cluster. see Secrets and Configuration
    • mod
      • description - a description of this mod
      • routes - an array of a source and destination JSON route modification. ex.[ "x.y", "x" ] overwrites "x" with "x.y".
  • localcontext- This defines what you consider the bare-metal environment you are in to be called. This is used for when calling wov-env-build --local.

Vascular Configuration

The WovTools Vascular Configuration is what enables all the tools to work together.

Its configuration is stored across global and project configuration files, the data it incorporates is stored in separate tools (Kubernetes, Git, Kops, ssh, Cloud Providers) and files (Secrets Files), it has its own scripts designed to support it, and it feeds the data in many forms to scripts, systems and tools to use for many purposes such as building, configuring, analyzing, running, versioning, testing, etc. These different sources of information are used to build Context Configurations.

Why "Vascular"? - The metaphor of a heart pumping blood to parts of the body is relevant to how WovTool's configuration system works.

Using Configuration Data

You may need to use configuration data in your own scripts. The values can be accessed in various ways:

  • List WovTools Environment: wov-env -e >> A=1\nB=2\n...
    • one-line: wov-env -E >> A=1 B=2 ...
    • add Cloud provider info wov-env -p -e >> A=1\nB=1\n...
  • List merged secrets files: wov-env --envs >> A=1\nB=1\n...`
    • one-line: wov-env --conf >> A=1 B=1 ...`
    • bash exports: wov-env --exports >> export A=1\nexport B=1\n...`
  • Get a variable: wov-env --var A >> A=1

Hierarchy to Environment Format

The Secrets Files are hierarchical and environment variables are not, so the JSON hierarchy is converted to underscores. So, { "A" : { "B" : 1 } } would be an environment variable of A_B=1.

Hierarchy to .Wov File Format

The Secrets Files are hierarchical and so are the variables in .Wov Files. So, { "A" : { "B" : 1 } } would be used in a .Wov file, separated by dots, such as follows '{{A.B}}+{{A.B}}=2' produces '1+1=2'.

Microservice Variables

Only the necessary variables are sent to Kubernetes ConfigMaps and Secrets, which runs project code to generate a list used by the wov-env cmd:

  • wov-env --cm X
  • wov-env --se X

In NodeJS, there is a 'PROJECT/MICROSERVICE/src/MICROSERVICEconfig.js' or 'PROJECT/src/MICROSERVICEconfig.js' file that is run. For other languages, in the same location, a '.sh' extension instead of a '.js' extension shows a bash script that echos the variables.

Building Context Configurations

In WovTools, this is all handled behind the scenes as each script checks for changes in the system or context and then builds a new Context Configuration if a dependency changed. It uses wov-env-build and stores the output in a TEAMPROJECT/wovtools/cache/clusters/ORIGIN__CONTEXT/context.json. The overall process is the merging of all Secret Files, applying patches from your project configuration and then accounting for WovTools context.

Secret Files

The files in the Secrets Local Archive (linked from TEAMPROJECT/wovtools/secrets) are multiple JSON files that store data. In general, your teams should organize these by having the default X.json and then an X_STAGE.json that has specifics for that stage. These files have information for: repositories, clusters, the master project, WovDatabases and microservices.

The format for secret files is JSON but additionally, include keys preempted by ORIGIN, STAGE and CLUSTER. This information is collapsed down and used or discarded, based upon your WovTools context. For example, below, the output of .foo.bar is 1, unless the WovTools context stage is dev, which would then be 2.

{
  "foo" : {
    "bar" : 1,
    "STAGEdev : { "bar" : 2 }
  }
}

Secret File Selection

A Context Configuration selects files based upon the Project Configuration's .secrets entry. It stores an array of files for each stage, and the files are appended in order.

Origin Mods

A Project Configuration also has .originmods that applies after the secrets files are merged. So, if you have a originmods entry in config.json, of "here : { routes : [ microservice.CLUSTERlocal, microservice ] }", and your cluster is "local", then it puts everything at microservice.CLUSTERlocal into microservice.

NOTE: I don't think this is used anymore and probably should be removed.

Example

In the following example, we should a microservice named 'apirest' and a WovDatabase of 'apirestdb'. The microservice 'apirest' uses node 10.1. However, for cluster wov-aws-va-dog it is 11.1 and the development stage is testing 12.1. In handlebars preprocessed files, these are accessed by apirest.containerfrom but environment variables would be WOV_apirest_containerfrom. For the database named apirestdb, the username is postgres by default, but apirestme for stage me, and apirestdev for stage dev. As well, if you are on your laptop developing locally (ORIGIN here), then port 5432 is used to connect to the default Postgres port. But, ORIGIN external connects to port 65432 which is an ssh tunnel port to the database in the cloud.

{
  "apirest" : {
    "healthpath" : "/api/v1/health",
    "ver" : "v1",
    "containerfrom"         : "node:10.1",
    "CLUSTERwov-aws-va-dog" : {
      "containerfrom"         : "node:11.1",
    },
    "STAGEdev" : {
      "containerfrom"         : "node:12.1",
    }
  },
  "apirestdb" : {
    "username" : "postgres",
    "STAGEme" : {"username" : "apirestme" },
    "STAGEdev" : {"username" : "apirestdev"},
    "CLUSTERwov-aws-va-grape" : { STAGEprod : {"password" : "foobar"}},
    "ORIGINhere"     : {"port" : "5432"},
    "ORIGINexternal" : {"port" : "65432"}
  }
}