Network Troubleshooting - q-uest/notes-doc-k8s-docker-jenkins-all-else GitHub Wiki

Check which Network interface (physical/virtual) is used for routing

To check which interface through which your traffic is routing through for the given IP:

>> ip route get 10.96.0.1

Output:

10.96.0.1 via 10.182.0.1 dev ens4 src 10.182.0.2 uid 1000
    cache

To get only the IP address of interface from the above command:

ip route get 10.96.0.1| awk '{print $7}'

Examples:

ip route get 142.251.2.102
142.251.2.102 via 10.182.0.1 dev ens4 src 10.182.0.2 uid 1000
    cache


ip route get 172.17.0.100
172.17.0.100 dev docker0 src 172.17.0.1 uid 1000
    cache

## Try K8s Pod's IP address

ip route get 10.244.1.16
10.244.1.16 via 10.244.1.0 dev flannel.1 src 10.244.0.0 uid 1000
    cache


# Try a K8s service's IP address:

 ip route get 10.105.202.197
 10.105.202.197 via 10.182.0.1 dev ens4 src 10.182.0.2 uid 1000
    cache


IP Routing Table with "netstat":

This is to get network statistics which is used to display very detailed information about how your computer is communicating with other computers or network devices.Specifically, it can show details about individual network connections, overall and protocol specific networking statistics, and much more, all of which could help troubleshoot certain kinds of networking issues.

netstat -r

netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
default         _gateway        0.0.0.0         UG        0 0          0 ens4
_gateway        0.0.0.0         255.255.255.255 UH        0 0          0 ens4
10.244.0.0      0.0.0.0         255.255.255.0   U         0 0          0 cni0
10.244.1.0      10.244.1.0      255.255.255.0   UG        0 0          0 flannel.1
172.17.0.0      0.0.0.0         255.255.0.0     U         0 0          0 docker0

Network Manager to get Device Details:

Displays Device Type, Device's IP address & Routes etc.

 nmcli -p device show
                             Device details (ens4)
===============================================================================
GENERAL.DEVICE:                         ens4
-------------------------------------------------------------------------------
GENERAL.TYPE:                           ethernet
-------------------------------------------------------------------------------
GENERAL.HWADDR:                         42:01:0A:B6:00:02
-------------------------------------------------------------------------------
GENERAL.MTU:                            1460
-------------------------------------------------------------------------------
GENERAL.STATE:                          10 (unmanaged)
-------------------------------------------------------------------------------
GENERAL.CONNECTION:                     --
-------------------------------------------------------------------------------
GENERAL.CON-PATH:                       --
-------------------------------------------------------------------------------
WIRED-PROPERTIES.CARRIER:               on
-------------------------------------------------------------------------------
IP4.ADDRESS[1]:                         10.182.0.2/32
IP4.GATEWAY:                            10.182.0.1
IP4.ROUTE[1]:                           dst = 0.0.0.0/0, nh = 10.182.0.1, mt = 100
IP4.ROUTE[2]:                           dst = 10.182.0.1/32, nh = 0.0.0.0, mt = 100
-------------------------------------------------------------------------------
IP6.ADDRESS[1]:                         fe80::4001:aff:feb6:2/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 256
-------------------------------------------------------------------------------

===============================================================================
                         Device details (veth4eae5bbe)
===============================================================================
GENERAL.DEVICE:                         veth4eae5bbe
-------------------------------------------------------------------------------
GENERAL.TYPE:                           ethernet
-------------------------------------------------------------------------------
GENERAL.HWADDR:                         EE:32:A4:E9:12:5D
-------------------------------------------------------------------------------
GENERAL.MTU:                            1410
-------------------------------------------------------------------------------
GENERAL.STATE:                          10 (unmanaged)
-------------------------------------------------------------------------------
GENERAL.CONNECTION:                     --
-------------------------------------------------------------------------------
GENERAL.CON-PATH:                       --
-------------------------------------------------------------------------------
WIRED-PROPERTIES.CARRIER:               on
-------------------------------------------------------------------------------

===============================================================================
                           Device details (docker0)
===============================================================================
GENERAL.DEVICE:                         docker0
-------------------------------------------------------------------------------
GENERAL.TYPE:                           bridge
-------------------------------------------------------------------------------
GENERAL.HWADDR:                         02:42:74:FA:7B:0C
-------------------------------------------------------------------------------
GENERAL.MTU:                            1500
-------------------------------------------------------------------------------
GENERAL.STATE:                          10 (unmanaged)
-------------------------------------------------------------------------------
GENERAL.CONNECTION:                     --
-------------------------------------------------------------------------------
GENERAL.CON-PATH:                       --
-------------------------------------------------------------------------------
IP4.ADDRESS[1]:                         172.17.0.1/16
IP4.GATEWAY:                            --
IP4.ROUTE[1]:                           dst = 172.17.0.0/16, nh = 0.0.0.0, mt = 0
-------------------------------------------------------------------------------
IP6.GATEWAY:                            --
-------------------------------------------------------------------------------

===============================================================================
                          Device details (flannel.1)
===============================================================================
GENERAL.DEVICE:                         flannel.1
-------------------------------------------------------------------------------
GENERAL.TYPE:                           vxlan
-------------------------------------------------------------------------------
GENERAL.HWADDR:                         9A:22:AE:3D:61:8F
-------------------------------------------------------------------------------
GENERAL.MTU:                            1410
-------------------------------------------------------------------------------
GENERAL.STATE:                          10 (unmanaged)
-------------------------------------------------------------------------------
GENERAL.CONNECTION:                     --
-------------------------------------------------------------------------------
GENERAL.CON-PATH:                       --
-------------------------------------------------------------------------------
IP4.ADDRESS[1]:                         10.244.0.0/32
IP4.GATEWAY:                            --
IP4.ROUTE[1]:                           dst = 10.244.1.0/24, nh = 10.244.1.0, mt = 0
-------------------------------------------------------------------------------
IP6.ADDRESS[1]:                         fe80::9822:aeff:fe3d:618f/64
IP6.GATEWAY:                            --
IP6.ROUTE[1]:                           dst = fe80::/64, nh = ::, mt = 256
-------------------------------------------------------------------------------



Configuration of ens4 interface

/etc/netplan/50-cloud-init.yaml:


# This file is generated from information provided by the datasource.  Changes
# to it will not persist across an instance reboot.  To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
    ethernets:
        ens4:
            dhcp4: true
            match:
                macaddress: 42:01:0a:b6:00:02
            set-name: ens4
    version: 2

=====

Why I do not get to see the ports being used by a Docker container in "netstat"? How to get it too?

use netstat with "All" and "Numeric" options as below:

sudo netstat -an

===

** How to access a Nginx pod without creating a service?**

>> kubectl get pod -o wide

NAME                               READY   STATUS    RESTARTS      AGE     IP                NODE       NOMINATED NODE   READINESS GATES
nginx-test                         1/1     Running   0             56m     192.168.161.242   k-node-1   <none>           <none>

The Pod is accessible from the Master node as below:

curl 192.168.161.242
<!DOCTYPE html>
<html>
<head>
<title>Welcome to nginx!</title>
<style>
    body {
        width: 35em;
        margin: 0 auto;
        font-family: Tahoma, Verdana, Arial, sans-serif;
    }
</style>
</head>
<body>
<h1>Welcome to nginx!</h1>
<p>If you see this page, the nginx web server is successfully installed and
working. Further configuration is required.</p>

<p>For online documentation and support please refer to
<a href="http://nginx.org/">nginx.org</a>.<br/>
Commercial support is available at
<a href="http://nginx.com/">nginx.com</a>.</p>

<p><em>Thank you for using nginx.</em></p>
</body>
</html>

** Port-forwarding in order to Access the Pod from internet **

kubectl port-forward --address 0.0.0.0 pod/nginx-test 80:80

Now, this appears in the "netstat -tulpn" output as below:

# netstat -tulpn

Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:80              0.0.0.0:*               LISTEN      29589/kubectl    
⚠️ **GitHub.com Fallback** ⚠️