Kubernetes istio - ghdrako/doc_snipets GitHub Wiki

• Release

• Extension OpeShift to istio

• https://rinormaloku.com/series/istio-around-everything-else/ Introduction in 12 parts

The Istio Service Mesh comprises a data plane and a control plane.

• Istio control plane will be installed on its own separate set of nodes. The control plane comprises istiod components as well as a few other Kubernetes configs, which, altogether, are responsible for managing and providing service discovery to the data plane, propagation of configuration related to security and traffic management, as well as providing and managing identity and certificates to data plane components.
• The data plane is another part of the Service Mesh that consists of Istio proxies deployed alongside the application container in the Pod. Istio proxies are basically Envoy. Envoy is an application-aware service proxy that mediates all network traffic between microservices, based on instructions from the control plane. Envoy also collects various metrics and reports back telemetry to various add-on tools.

Control plain

istiod

istiod is one of the Istio control plane components, providing service discovery, configuration, and certificate management. istiod unifies the functionality of older components (Pilot, Galley, and Citadel) into a single binary to provide simplified installation, operation, and monitoring, as well as seamless upgrades between various Istio versions.

kubectl get po -n istio-system
kubectl exec -it pod/istiod-65fc7cdd7-r95jk -n istio-system -- /bin/sh -c "ps -ef"
kubectl get pod/istiod-65fc7cdd7-r95jk -n istio-system -o json | jq '.spec.containers[].image'  # istiod image is based on pilot-discovery
kubectl get pod/details-v1-7d79f8b95d-5f4td -n bookinfons -o json|jq '.spec.containers[].image' # the sidecar container is created from proxyv2

Configuration watch

istiod watches Istio Custom Resource Definitions (CRDs) and any other Istio-related configuration being sent to the Kubernetes API server. Any such configuration is then processed and distributed internally to various subcomponents of istiod. You interact with Istio Service Mesh via the Kubernetes API server, but all interactions with Kubernetes API server are not necessarily destined for the Service Mesh.

istiod keeps an eye on various config resources, typically Kubernetes API server resources identified by certain characteristics such as labels, namespaces, annotations, and so on. These configuration updates are then intercepted, collected, and transformed into Istio-specific formats and distributed via the Mesh Configuration Protocol (MCP) to other components of istiod. istiod also implements configuration forwarding, which we will be looking at in later chapters when we do a multi-cluster installation of Istio. For now, let’s just say that istiod can also pass configurations to another istiod instance over MCP in both pull and push modes.

API validation

istiod also adds an admission controller to enforce the validation of Istio resources before they are accepted by the Kubernetes API server.

• The** mutating webhook** is responsible for augmenting the API calls for resources such as deployments by adding configuration for Istio sidecar injection.
• the validation webhook auto registers itself with the Kubernetes API server to be called for each incoming call for Istio CRDs. When such calls to add/update/delete Istio CRDs arrive at the Kubernetes API server, they are passed to the validation webhook, which then validates the incoming request and, based on the outcome of the validation, the API calls are accepted or rejected.

Istio Certificate Authority

Istio provides comprehensive security for all communication in the mesh. All Pods are assigned an identity through the Istio PKI with x.509 key/cert in Spifee Verifiable Identity Document (SVID) format. The Istio Certificate Authority (CA) is responsible for signing requests from node agents deployed along with istio-proxy. The Istio CA is built on top of Citadel and is responsible for approving and signing the Certificate signature requests(CSRs) sent by Istio node agents. The Istio CA also performs the rotation and revocation of certificates and keys. It offers pluggability of different CAs as well as the flexibility to use the Kubernetes CA. Some of the other functions and components of the Istio control plane are as follows:

• Sidecar injection: The Istio control plane also manages sidecar injection via mutating webhooks.
• Istio node agent: Node agents are deployed along with Envoy and take care of communication with the Istio CA, providing the cert and keys to Envoy.
• Identity directory and registry: The Istio control plane manages a directory of identities for various types of workloads that will be used by the Istio CA to issue key/certs for requested identities.
• End-user context propagation: Istio provides a secure mechanism to perform end user authentication on Ingress and then propagate the user context to other services and apps within the Service Mesh. The user context is propagated in JWT format, which helps to pass on user information to services within the mesh without needing to pass the end user credentials.

The Istio operator and istioctl

The Istio operator and istioctl are both control plane components and are optional to install. Both provide administrative functions to install and configure components of the control and data planes.

The Istio operator and istioctl essentially perform the same functions with the exception that istioctl is explicitly invoked to make a change, whereas the Istio operator functions per the operator framework/pattern of Kubernetes.

To install the Istio operator

$ istioctl operator init

To uninstall the Istio operator

$ istioctl operator remove 
$ kubectl delete ns istio-operator

The two main components of the Istio operator are the customer resource called IstioOperator, represented by high-level APIs, and a controller that has logic to transform the high-level API into low-level Kubernetes actions. The IstioOperator CRD wraps a second component called IstioOperatorSpec, a status field, and some additional metadata. You can use the following command to find details of the IstioOperator Custom Resource (CR):

$ kubectl get istiooperators.install.istio.io -n istio-system -o json

istioctl and the operator are very similar to each other except when in the Actuation phase. istioctl is a user-run command that takes an IstioOperator CR as input, while the controller runs whenever the in-cluster IstioOperator CR changes, but the remaining components are similar, if not the same.

The following is a brief summary of the various components of the Istio operator and istioctl:

• Actuation: Triggers the validator component in response to an event, for example, a request for a CR update. For istioctl, the actuation logic is triggered by the operator invoking the istioctl CLI, which is written in Go using Cobra, a library for creating powerful CLI applications.
• Validator: Verifies the input (the IstioOperator CR) against the original schema of the CR.
• Config generator: In this phase, a full-blown configuration is created. The configuration includes parameters and values provided in the original event, as well as parameters that were omitted in the original event. The configuration contains the omitted parameters, along with their respective default values.
• Translator and renderer: The translator maps IstioOperator’s Kubernetes resources specs to Kubernetes resources, while the renderer produces the output manifest after applying all configurations.
• Resource Manager: This is responsible for managing the resources in the cluster. It caches the recent state of resources in a built-in cache, which is then compared with the output manifests, and every time there is a deviation or inconsistency between the state of Kubernetes objects (namespaces, CRDs, ServiceAccounts, ClusterRoles, ClusterRoleBindings, MutatingWebhookConfigurations, ValidatingWebhookConfigurations, Services, Deployments, or ConfigMaps) and the output manifest, then the Resource Manager updates them as per the manifest.

Istio agent

The Istio agent (also called pilot-agent) is part of the control plane deployed in every istio-proxy to help connect to the mesh by securely passing configuration and secrets to the Envoy proxies.

$ kubectl exec -it details-v1-7d79f8b95d-5f4td -c istio-proxy -n bookinfons --/bin/sh -c "ps -ef"

Pilot-agent not only bootstraps the Envoy proxy but also generates key and certificate pairs for Envoy proxies to establish the identity of Envoy proxies during mesh communication.

Secret Discovery Service (SDS)

The SDS simplifies certificate management and was originally created by the Envoy project to provide a flexible API to deliver secrets/certificates to the Envoy proxy. The components needing the certificates are called SDS clients, and the component generating the certificates is called the SDS server. In the Istio data plane, the Envoy proxy acts as an SDS client and the Istio agent acts as the SDS server. The communication between the SDS client and SDS server happens using the SDS API specifications, mostly implemented over gRPC.

Step performed between the Istio agent, Envoy, and istiod to generate the certificate:

1. During sidecar injection, istiod passes information about the SDS, including the location of the SDS server to the Envoy proxy.
2. Envoy sends a request to pilot-agent (SDS server) for certificate generation over a Unix domain socket (UDS) via SDS protocols. pilot-agent generates a certificate signing request.
3. pilot-agent then communicates with istiod and provides its identity along with the certificate signing request.
4. istiod authenticates pilot-agent and if all is OK, signs the certificate.
5. pilot-agent passes the certificate and keys to the Envoy proxy over UDS.

Gateway is generally used to expose a VirtualService to the outside world. So with this object we can control how and which traffic from outside will reach one of our VirtualServices. It is also possible to specify how Gateway treats the traffic, E.g. TLS termination or SNI passthrough.

In short Gateway is for external traffic while VirtualService is for traffic that is already inside the istio cluster.

Destination rules are applied after virtual service routing rules are evaluated, so they apply to the traffic's “real” destination. In particular, you use destination rules to specify named service subsets, such as grouping all a given service's instances by version.

Istio Gateway

Ingress refers to traffic that originates outside the network and is intended for an endpoint within the network. The traffic is first routed to an ingress point that acts as a gatekeeper for traffic coming into the network. Hosting multiple different services at a single entry point is known as virtual hosting

virtual IP address that represents our service and forwards traffic to our actual service instances, provides us with higher-availability and flexibility. The virtual IP is bound to a type of ingress point known as a reverse proxy. The reverse proxy is an intermediary component that’s responsible for distributing requests to backend services and does not correspond to any specific service. The reverse proxy can also provide capabilities like load balancing so requests don’t overwhelm any one backend.

Istio uses a single Envoy proxy as the ingress gateway. The Gateway resource defines ports, protocols, and virtual hosts that we wish to listen for at the edge of our service-mesh cluster. VirtualService resources define where traffic should go once it’s allowed in at the edge. Istio Gateway handles the L4 and L5 concerns, while VirtualService handles the L7 concerns.

For production, you should run the ingress gateway component in its own namespace, separate from istio-system. In (Istio 1.13.0), the secret used for TLS in the gateway can only be retrieved if it’s in the same namespace as the Istio ingress gateway.

Istio’s gateway is really just a simple Envoy proxy that’s not deployed as a sidecar. This means you can use the gateway for various use cases such as ingress (covered here), egress, shared-gateway functionality, multi-cluster proxying, and so on.

TLS mode is configurable per host with one of the following modes:

• Encrypt and prevent man-in-the-middle attacks with the SIMPLE TLS mode.
• Mutually authenticate both server and client with the MUTUAL TLS mode.
• Admit and reverse proxy encrypted traffic using the SNI header with the PASSTHROUGH TLS mode.

kubectl -n istio-system exec \
deploy/istio-ingressgateway -- ps

Two Istio resources: Gateway and VirtualService are fundamental for getting traffic to flow in Istio.

kubectl get svc istio-ingressgateway -n istio-system
$ export INGRESS_HOST=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.status.loadBalancer.ingress[0].ip}')
$ export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].port}')
$ export SECURE_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="https")].port}')
$ export TCP_INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="tcp")].port}')

Configuring ingress using an Istio gateway

1. Create an Istio Gateway To configure an ingress gateway in Istio, we use the Gateway resource and specify which ports we wish to open and what virtual hosts to allow for those ports.

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: Gateway
metadata:
  name: httpbin-gateway     # Name of the gateway
spec:
  selector:
    istio: ingressgateway # use Istio default gateway implementation
  servers:
  - port:
      number: 80   # Ports to expose
      name: http
      protocol: HTTP
    hosts:
    - "httpbin.example.com"  # Host(s) for this port
EOF

We exposes an HTTP port on port 80 that accepts traffic destined for virtual host httpbin.example.com

Our Gateway resource configures Envoy to listen on port 80 and expect HTTP traffic.

istioctl -n istio-system proxy-config \
listener deploy/istio-ingressgateway


istioctl proxy-config route deploy/istio-ingressgateway \
-o json --name http.8080  -n istio-system

The Pod running the gateway, whether that’s the default istio-ingressgateway or your own custom gateway, must be able to listen on a port or IP that is exposed outside the cluster. If you’re deploying on a cloud service like Google Container Engine (GKE), make sure you use a service of type LoadBalancer, which gets an externally routable IP address.

2. Configure routes for traffic entering via the Gateway: So far, all we’ve done is configure an Istio gateway to expose a specific port, expect a specific protocol on that port, and define specific hosts to serve from the port/protocol pair. When traffic comes into the gateway, we need a way to get it to a specific service within the service mesh; and to do that, we’ll use the VirtualService resource. In Istio, a VirtualService resource defines how a client talks to a specific service through its fully qualified domain name, which versions of a service are available, and other routing properties (like retries and request timeouts).

$ kubectl apply -f - <<EOF
apiVersion: networking.istio.io/v1alpha3
kind: VirtualService
metadata:
  name: httpbin    # Name of the virtual service 
spec:
  hosts:
  - "httpbin.example.com"   # Virtual host name(s) to match
  gateways:
  - httpbin-gateway         # Gateways to which this applies
  http:
  - match:
    - uri:
        prefix: /status
    - uri:
        prefix: /delay
    route:
    - destination:         # Destination service for this traffic
        port:
          number: 8000
        host: httpbin
EOF

$ curl -s -I -HHost:httpbin.example.com "http://$INGRESS_HOST:$INGRESS_PORT/status/200"

Wo work in browser change - "httpbin.example.com" to - "*"

istioctl proxy-config route deploy/istio-ingressgateway \
-o json --name http.8080  -n istio-system

The Gateway resource defines ports, protocols, and virtual hosts that we wish to listen for at the edge of our service-mesh cluster. VirtualService resources define where traffic should go once it’s allowed in at the edge.

extensionProviders

extensionProviders | ExtensionProvider[] | Defines a list of extension providers that extend Istio’s functionality. For example, the AuthorizationPolicy can be used with an extension provider to delegate the authorization decision to a custom authorization system. extensionProviders ExtensionProvider[] Defines a list of extension providers that extend Istio’s functionality. For example, the AuthorizationPolicy can be used with an extension provider to delegate the authorization decision to a custom authorization system.

Parameters

https://istio.io/latest/docs/reference/config/istio.mesh.v1alpha1/

holdApplicationUntilProxyStarts | BoolValue | Boolean flag for enabling/disabling the holdApplicationUntilProxyStarts behavior. This feature adds hooks to delay application startup until the pod proxy is ready to accept traffic, mitigating some startup race conditions. Default value is ‘false’.

spec.meshConfig.defaultConfig.holdApplicationUntilProxyStarts is used as default setting

apiVersion: install.istio.io/v1alpha2
kind: IstioOperator
spec:
  meshConfig:
    defaultConfig:
      holdApplicationUntilProxyStarts: true

Pod annotation can override that.

annotations: proxy.istio.io/config: '{ "holdApplicationUntilProxyStarts": false }'

spec:
…
  template:
    metadata:
…
      annotations:
        proxy.istio.io/config: '{"holdApplicationUntilProxyStarts": true }'