< Home

Q. What's the WovTools special sauce?

A. WovTools is to Clusters as Docker is to containers.

A. If you want to get running fast with a microservice orchestrated cluster or run different clusters with different configurations and at different regions, WovTools manages the development, deploying to and switching between your single to many clusters.

Q. How expressive is WovTools? If I can do a lot, it's complex. If it's simple, it will limit my long-term product plan. Right?

A. That's the normal tradeoff, but WovTools is:

• at worst: as complex as current tools we use (git, Kubernetes, cloud CLI tools, database cli, Kops, etc); and
• at best: as good as our command line tools which optimize common DevInOps workflow.

Huh? We claim this because:

• Generally, the more expressive a tool/notation is, the more complex it becomes and the larger the barrier to entry. WovTools avoids this because it lets you continue to use your tools by on-the-fly configurations for your current cluster.
• In addition to on-the-fly configurations, we write boilerplate templated files for your common tools (Handlebars format mostly). For your special cases and long-term project plan optimizations, just modify the template files.
• Versioning of these files and configuration is matched to your code, database schemas and clusters so if it ran once upon a time, it will continue to do so.

Q. Where does this fit into my development process?

A. You develop code. You run it locally and it works. How do you get that into Kubernetes? This helps.

Q. How do you handle multiple developers on a project and Kubernetes?

A. Each developer has their own space in the cluster sealed off from the rest (Kubernetes namespaces) and their own Git branch to feed that namespace. The power and familiarity of merging Git branches is how you push to dev and production.

Q. Doesn't this mean a developer could impact production by too many pods?

A. Yes, by using too many resources in the cluster, but they can't screw up something in another namespace. If you can't live with that, deploy a small testing cluster for your development team. Heck, they could spin up a Kubernetes cluster on their dev machine before even sending to a remote or cloud cluster (How slick was that?).

Q. What does WovTools do to my development environment?

A. We create a 'wovtools' directory, where we place our files. We also modify your Git config to have 'dev' and 'prod' branches, with 'prod' being master. For databases, we ask you name them and we monitor for schema changes before pushing a version. If you have them, you will want to move your Kubernetes files into wovtools/k8s.

Q. How do you handle the massive amount of configuration, for local as well as running in the cluster?

A. Kubernetes is great once it runs. Getting something running locally requires a lot of work before it runs in a development stage of your Kubernetes cluster, and still more configuration changes when it rolls to production. We handle this... - ... by placing all your configuration data into json files (organize them as you like, they get merged together later anyway) (THE MERGING IS A SLICK AND EXPRESSIVE NON-TRIVIAL BUT INTUITIVE PROCESS); - ... by instrumenting configuration files (environment variables and k8s yaml) with two-pass handlebars expresssions, with the first pass converting expressions to the appropriate stage; - ... by maintaining an up-to-date configuration environment (we rebuild the configuration as necessary with wov-build); and - ...by versioning the configuration (WOV_SVER) and use it to generate Kubernetes ConfigMaps and Secrets for running pods.

Q. Wait, how do you manage configuration?

A. TL;DR: You don't have separate configuration environments, you just change your stage (the wov-stage command is a curses GUI to toggle this). This changes the Kubernetes namespace, your Git branch and updates your configured settings.

A. Every script, system, pod, blah... needs some configuration. It's a nightmare. So, we use JSON as the source data, merging multiple files together and collapsing branches based upon your CLUSTER, ORIGIN and STAGE. (So, your global project configuration is baseline and you add files specific to your stage, your laptop, a specific cluster, etc. Changing stages (to dev, prod or to another user) changes how the branches collapse and lead to a new configuration.). We use this data to generate environment variables and process templated files, with updates to the Handlebars format that select between CLUSTER, ORIGIN or STAGE as well. Then, for access control in your organization, you restrict who has which json files so only certain people have dev and prod stage access.

Q. Can you give an example of how configuration works? Here we have a mysecret.json file which generates two files, one for code (config.ck8s) and one for secrets (config.sk8s). They are loaded into two different scripts (healthcheck.sh and getdata.sh) with the wov-env command. Now, if you need to call 'healthcheck.sh', it never needs to know about your Kubernetes context (and wov-env does a wov-build check so never becomes out of sync).

# NOTE: my initials are 'cw' so I am using 'cw' to represent my personal stage of development (used in Kubernetes namespace and git branch)
# FILE wovtools/secrets/myproject.json
{
  "my": {
      "cw"       : {"url": "my-cw.site.com", "token": "IO0IExCECXpK"},
      "cw-local" : {"url": "localhost:8080", "token": "Y4IIgJZZdYMh"},
      "dev"      : {"url": "my-dev.site.com", "token": "FvdRWwRLD9Tq"},
      "prod"     : {"url": "my.site.com"}, "token": "lDLMNXFoUmsuFddX6D" }
}

# FILE wovtools/conf/config.ck8s
# regular environment variables go here
MY_SITE_URL=\{{my.{{STAGE}}.url}}

<${WOV_BASE}/wovtools/conf/config.sk8s>
# sensitive environment variables go here
MY_SITE_TOKEN=\{{my.{{STAGE}}.token}}

<healthcheck.sh>
. $(wov-env --env config.ck8s)
curl ${MY_SITE_URL}/api/v1/health

<getdata.sh>
. $(wov-env --conf) # equivalent to $(wov-env --env config.ck8s --env config.sk8s)
curl ${MY_SITE_URL}/api/v1/foo&token=${MY_SITE_TOKEN}

Q. How do you manage secrets?

A. We use Git to versions secrets. This gives us a SVER number which we pair with the PVER to create a label when pushing content to the Archive.

NOTE: So, PLEASE make sure you trust your repository location (i.e. Github (and likely other free services) are not secure). Also, our preprocessing data is pushed to a projectroot/wovtools/cache directory restricted to only the user. You should probably encrypt your harddrive for another layer of secrity if someone goes around this restriction.

Q. How to you manage databases?

A. We roughly handle that. Databases are data and schema (SQL at least) so it can be tricky. But, we track changes to schema and loading of snapshots. If we detect changes to a db schema (checksum of database schema dump) when you are pushing, their better be a database delta file to account for that change. These delta files get pushed into the achive. We also have a command to log in to Postgres and MongoDB databases based upon their deployment name (wov-db-connect <mydatabase>). All this is managed with the configuration system.

Q. What tools does this use?

A. Kubernetes, Docker and git are the main tools this uses. It was developed while developing with NodeJS for microservices.