mTLS - TheOpenCloudEngine/uEngine-cloud GitHub Wiki

Mutual TLS and Istio

Important
Before Start

You should have NO virtualservice, destinationrule, gateway or policy (in tutorial namespace) istioctl get virtualservice istioctl get destinationrule istioctl get gateway istioctl get policy if so run:

./scripts/clean.sh

In this chapter, we are going to see how to secure the communication between all services by enabling mutual TLS in Istio.

Before Start

If you are running this tutorial in OpenShift (Minishift) you need to remove the customer route you created at the beginning of the tutorial.

To do it just run:

oc delete route customer

Then, you need to enable Istio Ingress to receive all traffic and redirect it to customer service. For this reason, let’s create a Gateway and VirtualService that allows local calls reach the clustered service inside the mesh.

istioctl create -f istiofiles/gateway-customer.yml

Then configure some environment variables that will help you on testing:

export INGRESS_PORT=$(oc -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')

or in Kubernetes:

export INGRESS_PORT=$(kubectl -n istio-system get service istio-ingressgateway -o jsonpath='{.spec.ports[?(@.name=="http2")].nodePort}')
export INGRESS_HOST=$(minishift ip)
export GATEWAY_URL=$INGRESS_HOST:$INGRESS_PORT

Now you can run curl but against GATEWAY_URL and you’ll see the same messages as before:

curl http://${GATEWAY_URL}/
customer => preference => recommendation v1 from 'b87789c58-mfrhr': 2

Enabling mTLS

Now that Istio gateway is in place, you can enable mTLS by applying next Istio resources:

istioctl create -f istiofiles/authentication-enable-tls.yml # (1)
istioctl create -f istiofiles/destination-rule-tls.yml # (2)

  1. Enables mTLS into tutorial namespace

  2. Makes that services withn tutorial namespace communicates with mTLS

Then you can run:

curl http://${GATEWAY_URL}/
customer => preference => recommendation v1 from 'b87789c58-mfrhr': 2

And you’ll see exactly the same but now the communication between the services has been secured. How can you check that?

First running next command:

istioctl authn tls-check | grep tutorial

customer.tutorial.svc.cluster.local:8080                         OK         mTLS       mTLS       default/tutorial                             default/tutorial
preference.tutorial.svc.cluster.local:8080                       OK         mTLS       mTLS       default/tutorial                             default/tutorial
recommendation.tutorial.svc.cluster.local:8080                   OK         mTLS       mTLS       default/tutorial                             default/tutorial

Second, by sniffing traffic between services, which is a bit more tedious, open a new terminal tab and run next command:

CUSTOMER_POD=$(oc get pod | grep cust | awk '{ print $1}' ) # (1)

or in Kuberentes:

CUSTOMER_POD=$(kubectl get pod | grep cust | awk '{ print $1}' )


oc exec -it $CUSTOMER_POD -c istio-proxy /bin/bash # (2)

or in Kubernetes:

kubectl exec -it $CUSTOMER_POD -c istio-proxy /bin/bash

# Inside pod shell

ifconfig # (3)
sudo tcpdump -vvvv -A -i eth0 '((dst port 8080) and (net 172.17.0.10))' # (4)

  1. Get customer pod name

  2. Open a shell inside pod

  3. Get IP of current pod (probably the IP represented at eth0 interface)

  4. Capture traffic from eth0 (or your interface) of port 8080 and network 172.17.0.10 (your IP from ifconfig)

Now all communication that happens between customer service and preference service is dumped in the console.

So now go to a terminal and execute:

curl http://${GATEWAY_URL}/
customer => preference => recommendation v1 from 'b87789c58-mfrhr': 2

Obviously, the response is exactly the same, but if you go to the terminal where you are executing tcpdump, you should see something like:

14:24:55.078222 IP (tos 0x0, ttl 64, id 32578, offset 0, flags [DF], proto TCP (6), length 967)
    172.17.0.15.33260 > customer-7dcd544ff9-652ds.8080: Flags [P.], cksum 0x5bf5 (incorrect -> 0x595e), seq 2211080917:2211081832, ack 2232186801, win 391, options [nop,nop,TS val 5958433 ecr 5779275], length 915: HTTP
E....B@.@._........
......j...w.....[......
.Z.!.X/K.............w$.?....&T.`n.....UX.C&)Cj....y..{.&..I..	..<.
.....A..q.;...o.9+.4..;...6|".......M.4Wm.:}.....^..v..2..?VW[&s........@}.~B.>D.k..H...r.... .L..i,.
...=..=..y..[.k..g..0..5.f%..vz|..t.....%.`.|...B..%r0.^k.y.....y.@l$O.....?...J..qc&.........Z.^&..F.....w.">7..	...[.......2.&........>......s.....5
.n$X.....l.#...... ..Q..u..jBI.Z.Eb$9.$.._...!.........~"Xx<....);........Z.
.y/E]......K......... [email protected].\.
.i.v...#.O<..^.F.	...?..:s...).....e......*..F.Kz..i.jk..xx...#....|[email protected].....*...l.v..G)T...9...M.....i.H ..=	.a.hp..&8..L..`.s..d_o.~.T ./.......9..	;F81.......S.{.....1rE..o...`..............c+U...}.{7..Y....Q4.#..(.c]Q...[..8..$u.b...=..6.....~..9..H....R
.9x*q....h0......O......q..Fb)..E..m..=.M.....W.Yk>.......;.2eys..E.....=q.;.k ....R.f.(./^F....4.c..*Y.4....es.....TX`nh..L.z.6....(.X.>c.V.0z........GF%.%..l4P.......@.^Q........46.g.#.n...e.k.._..>.T+.S...t}....

Notice that you cannot see any detail about the communication since it happened through TLS.

Now, let’s disable TLS:

istioctl delete -f istiofiles/authentication-enable-tls.yml
istioctl delete -f istiofiles/destination-rule-tls.yml

And execute again:

curl http://${GATEWAY_URL}/
customer => preference => recommendation v1 from 'b87789c58-mfrhr': 2

And again check tcpdump output:

host: 0192.168.64.70:31380
user-agent: curl/7.54.0
accept: */*
x-forwarded-for: 172.17.0.1
x-forwarded-proto: http
x-envoy-internal: true
x-request-id: e5c0b90f-341b-9edc-ac3e-7dd8b33f0e8b
x-envoy-decorator-operation: customer.tutorial.svc.cluster.local:8080/
x-b3-traceid: ce289e960a639d11
x-b3-spanid: ce289e960a639d11
x-b3-sampled: 1

Now, you can see that since there is no TLS enabled, the information is not shadowed but in clear.

Clean Up

source,bash]

istioctl delete -f istiofiles/gateway-customer.yml

or you can run:

./scripts/clean.sh

And finally in case of running in OpenShift (Minishift) you can restore the route:

oc expose service customer
⚠️ **GitHub.com Fallback** ⚠️