configuration - anongitmous/k8sShell GitHub Wiki

ℹ️ The cmdlets used to control the K8sShell configuration are here.

Overview

  • Each time a cmdlet is invoked, an attempt is made to resolve the kube config file, the kube context, and for those cmdlets that are namespaced, the namespace. These values can be configured in a variety of ways, which are hierarchical.
  • The most flexible way is via the persisted configuration path which involves a json file named whatever you choose that is passed to the Set–ClusterConfig cmdlet, e.g.:
    Set-ClusterConfig config_file_key -PersistedConfig .\config.json
  • In the previous, the value of config_file_key will be used as an index into the cluster-configs element within config.json and will result in the values of various runtime settings such as the kubernetes configuration file to use, the context, default namespace, etc. being loaded into memory.
  • Once a cluster config for a specific cluster has been loaded, then the kube config, kube context, and namespace arguments will not need to be supplied to any cmdlets invoked subsequent to the call to Set–ClusterConfig.
  • Any of the kube config, kube context, and namespace values can be overridden on an individual cmdlet basis via -KubeConfig (alias kc), -Context (alias ctx), and -Namespace (alias ns), respectively. e.g.
    • If Set–ClusterConfig had been called specifying a key of 'Ex1' and the config file had valid values for kube config, kube context, and namespace, then a call to Get–Pods could be made without the need to supply any of those values explicitly.
    • However, any of those values could be overridden for a given invocation of that cmdlet:
      Get-Pods -KubeConfig 'altCfg' -Context 'altCtx' -Namespace 'altNs'.
      The alternate values would only be in effect for that cmdlet's invocation.
  • Additionally, an entirely different cluster configuration can be loaded for a particular cmdlet's invocation via the -ClusterConfigKey (alias cck) parameter. e.g.
    • If Set–ClusterConfig had been called to load the kube config, kube context, and namespace values for cluster 'Ex1', then a call to Get–Pods could be made without the need to supply any of those values.
    • However, all of those values could be overridden (along with other cluster-specific runtime settings) by calling Get–Pods with the -ClusterConfigKey parameter and a valid key into the cluster configuration dictionary loaded into memory via Set–ClusterConfig:
      Get-Pods -ClusterConfigKey 'Ex2'.
      The alternate values would only be in effect for that cmdlet's invocation.

How the Kube Config File is Resolved

  1. If the user has provided a cluster key to a cmdlet via -ClusterConfigKey into an already-loaded configuration file, then the value for the key kube-config-file from the corresponding cluster config entry will be used as the kube config file.
  2. The kube config file passed to the cmdlet on the command line via -KubeConfig.
  3. If there is a session variable defined with the name KubeConfigFile, then its value will be used as the kube config file.
    a. When a cluster config from a loaded configuration file is active, this is the kube config file that is considered to be active.
    b. Hence, a cmdlet like Get–Pods can be run without specifying any other parameters as long as the cluster config either specifies a context, or the kube config file has a default context defined.
  4. If there is an environment variable named KUBECONFIG defined, then its value will be used as the kube config file.
  5. If a kube config file named config can be found in the $HOMEDRIVE$HOMEPATH\.kube\ directory, then that will be used as the kube config file.
    a. If this fails, then the command will fail.

How the Kube Context is Resolved

  1. If the user has provided a cluster key to a cmdlet via -ClusterConfigKey into an already-loaded configuration file, then the value for the key kube-context from the corresponding cluster config entry will be used as the kube config context.
  2. The context string passed to the cmdlet on the command line via -Context.
  3. If there is a session variable defined with the name KubeContext, then its value will be used as the kube config context.
    a. When a cluster config from a loaded configuration file is active, this is the kube config context that will be used.
    b. Hence, a cmdlet like Get–Pods can be run without specifying any other parameters.
  4. The default context defined in the kube config file.
    a. If this fails, then the command will fail.

How the Namespace is Resolved

  1. If the user has provided a cluster key to a cmdlet via -ClusterConfigKey into an already-loaded configuration file, then the value for the key kube-namespace from the corresponding cluster config entry will be used as the cluster namespace.
  2. The namespace string passed to the cmdlet on the command line via -Namespace.
  3. If there is a session variable defined with the name KubeNamespace, then its value will be used as the cluster namespace.
    a. When a cluster config from a loaded configuration file is active, this is the cluster namespace that will be used.
    b. Hence, a cmdlet like Get–Pods can be run without specifying any other parameters.
    c. If this fails, then the cmdlet will be run against all namespaces.

How the Output Directory is Resolved

  • Certain cmdlets generate text/log output, e.g. Save–Logs.
  • There is the capability to specify a cluster-agnostic output directory as well as having output directories per cluster.
  • The cluster-agnostic output directory is used in the event that there is no cluster-specific directory defined.
  1. If the user has provided a cluster key to a cmdlet via -ClusterConfigKey into an already-loaded configuration file, then the value for the key output-directory from the corresponding cluster config entry will be used as the cluster output directory.
  2. The output directory path string passed to the cmdlet on the command line via -Directory.
  3. If there is a session variable defined with the name OutputDirectory, then its value will be used as the output directory.
    a. When a cluster config from a loaded configuration file is active, this is the output directory that will be used.
    b. Hence, a cmdlet like Save–Logs can be run without providing the -Directory parameter.

Port Forwarding Particulars

  • The kube-port-forwarding name/value in the cluster configuration value offers the ability to match pods with local and remote port settings.
  • It depends upon the existence of pod label keys and values.
  • Looking at the sample config.json at the bottom of this page in the kube-port-forwarding section, we see e.g. a dictionary key of redis. This would be resolved as follows:
    1. A pod's labels would be examined as to whether there exists a label with a key that matches in a case-insensitive way the value for the key label-key. In this case, the value would be app.
    2. Check whether the value for the pod label app is a regex match for the dictionary key of redis. e.g. if the pod had a label named app with a value of redis-0, then there would be a match.
    3. Upon finding a match, a check is made as to whether a port forwarding has already occurred for a pod that regex matches e.g. redis.
      a. If so, then the value specified in local-port is incremented, and that value is used as the local port.
      b. If not, then the value specified in local-port is used as-is, and the value is stored.
    4. The remote port(s) to be used are set to the value(s) specified by remote-ports.
  • The above is neither intended to be nor in reality fullproof. If one were to forward ports to hundreds of pods at a time, then there could be collisions.
    • For most use cases, the above algorithm is quite simple and reliable.
    • The author has had a dozen or more ports forwarded to pods with no issues.
    • If it turns out that users are regularly forwarding ports to hundreds of pods where the increment of the local port for one pod label/key match runs into the starting local port for another pod's label/key match, then the above algorithm will need to be changed.

Pipeline Particulars

  • Each time a cmdlet is invoked, an InvocationConfig is created which is particular to that cmdlet's invocation which is a result either of the active cluster configuration or what is passed to a cmdlet on the command line, such as a kube config file, kube context, or namespace.
  • In the case of cmdlets which accept pipelined objects, the kube config file, kube context, and where applicable, the namespace of the pipelined object, take precedence over what is passed to the cmdlet taking the pipelined input.
    • Hence, if you pipe a node object into a pod object where the kube config files are different (e.g. Get-Nodes -KubeConfig 'foo.cfg' | Get-Pods -KubeConfig 'bar.cfg'), the kube config file specified for Get-Nodes would take priority in the invocation of Get-Pods.

Example

Config.json
{
  "kubectl-path": "C:/Kube/kubectl.exe",
  "kubectl-shell": "bash",
  "output-directory":  "C:/temp",
  "cluster-configs": {
    "production": {
      "cmdlet-directives": {
        "override-required": [
          "Remove-K8sJob"
        ],
        "forbidden": [
          "Remove-Deployment",
          "Remove-Endpoint",
          "Remove-Node",
          "Remove-PersistentVolumeClaim",
          "Remove-PersistentVolume",
          "Remove-Pod",
          "Remove-Secret",
          "Remove-K8sService"
        ]
      },
      "kube-config-file": "c:/kube/productioncfg",
      "kube-context": "us-west-17:12345678:cluster/dev-us-west-3a",
      "kube-namespace": "release",
      "kube-port-forwarding": {
        "mysql": {
          "label-key": "app",
          "local-port": 20306,
          "remote-ports": [ 3306 ]
        },
        "redis": {
          "label-key": "app",
          "local-port": 23379,
          "remote-ports": [ 6379 ]
        },
        "webapp": {
          "label-key": "app",
          "local-port": 8080,
          "remote-ports": [ 8080 ]
        }
      },
      "kube-proxy-port": null,
      "kubectl-shell": "bin/sh",
      "output-directory": null
    },
    "staging": {
      "cmdlet-directives": {
        "override-required": [
          "Remove-K8sJob"
        ],
        "forbidden": [
          "Remove-Deployment",
          "Remove-Endpoint",
          "Remove-Node",
          "Remove-PersistentVolumeClaim",
          "Remove-PersistentVolume",
          "Remove-Pod",
          "Remove-Secret",
          "Remove-K8sService"
        ]
      },
      "kube-context": "us-east-14:12345678:cluster/dev-us-east-2a",
      "kube-config-file": "c:/kube/othercfg",
      "kube-namespace": null,
      "kube-port-forwarding": {
        "mysql": {
          "label-key": "app",
          "local-port": 20306,
          "remote-ports": [ 3306 ]
        },
        "redis": {
          "label-key": "app",
          "local-port": 23379,
          "remote-ports": [ 6379 ]
        },
        "someservice-[2|3]": {
          "label-key": "app",
          "local-port": 35000,
          "remote-ports": [ 5000 ]
        },
        "webapp": {
          "label-key": "app",
          "local-port": 8080,
          "remote-ports": [ 8080 ]
        }
      },
      "kube-proxy-port": 8080,
      "kubectl-shell": null,
      "output-directory": null
    },
    "qa1": {
      "cmdlet-directives": {
        "override-required": [],
        "forbidden": []
      },
      "kube-config-file": "c:/kube/othercfg",
      "kube-context": "qa",
      "kube-namespace": "default",
      "kube-port-forwarding": {
        "mysql": {
          "label-key": "app",
          "local-port": 20306,
          "remote-ports": [ 3306 ]
        },
        "newservice": {
          "label-key": "app",
          "local-port": 27000,
          "remote-ports": [ 7000 ]
        },
        "redis": {
          "label-key": "app",
          "local-port": 23379,
          "remote-ports": [ 6379 ]
        },
        "someservice-[2|3]": {
          "label-key": "app",
          "local-port": 35000,
          "remote-ports": [ 5000 ]
        },
        "webapp": {
          "label-key": "app",
          "local-port": 8080,
          "remote-ports": [ 8080 ]
        }
      },
      "kube-proxy-port": null,
      "kubectl-shell": null,
      "output-directory": null
    }
  }
}
⚠️ **GitHub.com Fallback** ⚠️