Kubernetes istio routing virtual service - ghdrako/doc_snipets GitHub Wiki

Istio routing requires names for ports that are defined in the Kubernetes services in the <protocol>[<-suffix>] format. The following are the Istio-supported protocols:

• http
• http2
• https
• grpc
• mysql
• mongo
• redis
• tcp
• tls
• udp

Different services can have the same name for their ports, but the same service can’t have identical names for different ports . It is important to note that this is a naming convention followed by Istio and does not add any additional protocol values to the Kubernetes specification.

Istio will do routing based on application versions. To select nodes of a version, they must be labeled accordingly. Thus, all our deployments and pods must have app and version labels applied. Istio also uses these labels in metrics and telemetry data collection.

Istio routing can be applied only if the port exposed by the pod is declared in the deployment template. Ports can be declared as a list for the containerPort field in the deployment template. According to the Kubernetes documentation, the containerPort field is for information purposes. A container can run a service listening on 0.0.0.0 and a port. This port will be accessible from all containers in the cluster. Istio routing will be bypassed if it is applicable to ports that are not part of the deployment template.

Request routing is configured in a service mesh using the VirtualService and DestinationRule components.

DestinationRule resolves a request destination location into a network address in the Kubernetes cluster. Istio prescribes version numbers to be part of pod labels. These labels can then be matched in DestinationRule to define a version-based service subset for request handling.

The DestinationRule component is supposed to be configured by service owners. The service owners not only can create distinct subsets but also can define connectionPool, load balancing, and outlier detection attributes for each of them. These settings determine how the consumer services connect to each of the respective nodes.

In a service mesh where we have enabled a TLS handshake, the cost for every new connection is relatively high. Traditionally, we have added various connection pool drivers to our consumer application. But Istio provides out-of-the-box support for connection pools. The connectionPool configuration determines how a consumer service connects to this provider service. These settings must be fine-tuned by service owners. The connection pool settings are applicable to each host of the consumer service. Istio connection pooling supports the keepAlive TCP method. Thus, not only can we use a pool but can reuse unused connections. The settings have a separate set of attributes to configure HTTP and TCP connection pools. These attributes enable us to fine-tune the HTTP connection reuse. The following are the most important attributes. We will not cover all of the attributes; refer to the Istio docs to learn more.

• maxConnections: This setting defines the upper limit for the number of connections to the service. The default value is set to 1024.This setting is applicable to TCP and HTTPv1.0 services.
• connectionTimeout: This setting defines the TCP connection timeout.
• Http2MaxRequets: This setting is applicable to HTTPv2.0. In HTTP 2.0 we make a single connection and reuse it for multiple requests. The settings define the upper limit for the number of requests that can be performed over a connection.
• Http1MaxPendingRequests: This setting defines the upper limit for the number of HTTP requests pending over a connection. This is also applicable to HTTPv2.0/GRPC services.

We can configure connectionPool attributes for each of the defined subsets.

It is important to note that the connection pool is monitored by the Envoy proxy. Envoy initiates the following circuit breakers if the configured limits are breached:

• upstream_cx_overflow: This circuit breaker is thrown when a service in the cluster breaches the maximum number of connections. This is often applicable to TCP and HTTP/1 services. Since HTTP/2 reuses the same connection, the limit is not applicable to it.
• upstream_rq_pending_overflow: This circuit breaker is thrown when a service in the cluster makes more HTTP requests than the configured limits. This is often applicable to HTTP/2.
• upstream_rq_retry_overflow: This circuit breaker is thrown when a service in the cluster makes more HTTP requests than the configured limits.

Load balancing is the process of distributing requests among the different hosts of the selected destination. There are various mechanisms to achieve this. It can be configured with the loadBalancer settings. Istio supports the following types of load balancers:

• Round robin: This selects a host at random. This performs better if there is no health checking enabled for the selected pods.
• Least connection: The method performs O(1) lookup to determine two healthy hosts. It picks the one that is serving the least number of connections.
• Random: The method selects a host randomly. It performs better if there is no health checking enabled for the selected pods.
• Consistent hash: This method configures hashing based on request headers or cookies.

Outlier detection is the process of determining unhealthy hosts in a load- balanced cluster. The process is then followed by removing the hosts from the load-balanced set. In Istio, Envoy circuit breakers are used to keep track of errors caused by the destination host. These errors can be caused by the service or the respective sidecar. In both cases, the host will be marked as unhealthy.

Istio only records consecutive errors thrown by a service. The default value is set to five consecutive errors. For TCP-based services, connection timeouts are counted as errors. While HTTP-based services, 5xx HTTP responses are also recorded as errors. When recording these errors, Istio, by default, evicts a service for 30 seconds from the load-balanced set. After the elapsed interval, the host is back in the load-balanced set and is re- evaluated at an interval of 10 seconds (by default). These timings can be altered by configuring the various attributes available.

In summary, we have configured subsets in DestinationRule . The subsets select nodes by matching the configured selectors. Istio then applies the connectionPool, load balancing, and outlier detection settings to them. These settings can be configured at the DestinationRule level. The settings will then be applied to each and every subset created under it. But if there is any configuration at the subset level, it will then override the DestinationRule-level configuration.

The Istio VirtualService component has a behavior that is similar to the Service component in Kubernetes. Basically a VirtualService is an abstraction that maps a request to a service defined in the service mesh. The service can be either a Kubernetes service or a service defined by Istio. The destination resolution by a VirtualService component is performed using DestinationRule. A VirtualService component can perform destination resolution to handle the following use cases:

• Single version of the service
• Lookup based on HTTP headers to select a service or query parameters
• Weighted comparison between a set of selected service versions
• Define timeouts – that is, if a response is not received from the upstream service in X seconds, then the request should time out
• Retry – that is, how many times a request should be attempted if the upstream system is not responding or is too slow to respond

The VirtualService abstraction decouples request routing from application deployment. Thus, in a cluster, we can deploy many versions of the same service and distribute the load among them in a finely controlled manner.

Following the virtual service definition, we also need to define destination rules. Destination rules are a set of rules applied to the traffic after they have gone through the virtual service routing rules.

apiVersion: networking.istio.io/v1alpha3
kind: DestinationRule
metadata:
  name: envoy-destination
  namespace: chapter4
spec:
  host: envoy-dummy-svc
  subsets:
  - name: v1
    labels:
      version: v1
  - name: v2
    labels:
      version: v2

Virtual service is also capable of performing request rewrites. This behavior is configured using the following attributes:

• The match attribute defines which requests will perform the rewrite. The matching can be based on a URI, HTTP headers, query parameters, an HTTP method, scheme, etc. To perform the rewrite, we must specify the URI, along with other selectors (if required).
• The rewrite attribute defines the new URI patch to which the request needs to be sent. Depending on the type of match, the rewrite will replace only the matching URI part. This means if we are matching the URI prefix, then the rewrite will only change the prefix. If the complete URI is matched, then a rewrite will change the complete URI.
• The subset attribute defines the destination host to which the rewritten request is forwarded.

Istio is capable of performing an HTTP attributes lookup. The match attribute supports URIs, HTTP headers, query parameters, HTTP methods, schemes, etc. We can match any of the previously mentioned attributes and forward the request to a matching host.

An Istio configuration takes a string for all configuration parts of the match attribute. String likes true, TRUE, 25, etc., are converted to the appropriate data type and thus can’t be directly passed. These values can be converted to strings by enclosing them in double quotes, as we have done in the previous configuration.

Istio has the capability to distribute requests among various versions of a service in a configured ratio . The ratio is determined by the weight attribute of a destination.

A canary release is the process of releasing software to a subset of users. The process allows developers to validate the new version with a subset of users before rolling it out to the entire user base. If there are issues found with the new version, the release can be rolled back to a smaller set of servers. This helps in mitigating the impact and improving service uptime. Kubernetes also supports canary testing by managing the instance/ replication counts of the application. But this process of managing a pod instance quickly becomes complicated and difficult to support. Istio, on the other hand, has rich support for selecting requests and thus can do the job quite easily.

A canary release is supplementary to the blue-green deployment discussed in the first chapter. As a general process, the following steps are undertaken:

In the previous sections, we discussed simple request matching. The match method validates a simple attribute and does request forwarding. But this is not good enough for a fairly advanced use case like a canary release. The matching needs to handle multiple clauses joined together with operations like AND/OR. The Istio match attribute supports both these operations in the following manner:

• The AND operation is performed by nesting multiple conditions under a single match attribute.
• The OR operation is performed by having separate conditions under a single match attribute.

Traffic mirroring is another important feature that allows you to asynchronously copy traffic being sent to an upstream to another upstream service as well, also known as mirrored service. Traffic mirroring is on a fire-and-forget basis, where the sidecar/gateway will not wait for responses from the mirrored upstream.

There are very interesting use cases for traffic mirroring, including the following:

• Traffic mirroring to pre-production systems for testing purposes
• Traffic mirroring to sink systems where traffic is recorded for out-of-band analysis

In the following example, in the virtual service definition under route configuration, we are mentioning that 100% of traffic should be mirrored to subset: v2:

route:
  - destination:
      port:
        number: 80
        subset: v1
        host: envoydummy
        weight: 100
  mirror:
    host: nginxdummy
    subset: v2
  mirrorPercentage:
    value: 100.0