Cluster&APP Health - Almax84/consulting-wiki GitHub Wiki

oc get nodes
Displays a column with the status of each node. If a node is not Ready, then it cannot communicate with the OpenShift control plane, and is effectively dead to the cluster.

oc adm top nodes
Displays the current CPU and memory usage of each node. These are actual usage numbers, not the resource requests that the OpenShift scheduler considers as the available and used capacity of the node.

oc describe node my-node-name
Displays the resources available and used from the scheduler point of view, and other information. Look for the headings "Capacity," "Allocatable," and "Allocated resources" in the output. The heading "Conditions" indicates whether the node is under memory pressure, disk pressure, or some other condition that would prevent the node from starting new containers.

oc get clusterversion
The output lists the version, including minor releases, the cluster uptime for a given version, and the overall status of the cluster.

oc describe clusterversion
More detailed output

oc get clusteroperators
to retrieve the list of all cluster operators

DISPLAYING THE LOGS OF A NODE

An OpenShift node based on Red Hat Enterprise Linux CoreOS runs very few local services that would require direct access to a node to inspect their status. Most of the system services in Red Hat Enterprise Linux CoreOS run as containers. The main exceptions are the CRI-O container engine and the Kubelet, which are Systemd units. To view these logs, use the oc adm node-logs command as shown in the following examples:

[user@host ~]$ oc adm node-logs -u crio my-node-name

[user@host ~]$ oc adm node-logs -u kubelet my-node-name

You can also display all journal logs of a node:

[user@host ~]$ oc adm node-logs my-node-name

Opening a Shell Prompt on an OpenShift Node

[user@host ~]$ oc debug node/my-node-name ...output omitted... sh-4.4# chroot /host sh-4.4# systemctl is-active kubelet active

APPLICATION LOGS & DEBUG

oc logs my-pod-name

If the pods contain multiple containers
oc logs my-pod-name -c my-container-name

Create troubleshooting pods:
oc debug deployment/my-deployment-name --as-root

Connect to the pod: oc rsh my-pod-name

Copy file from/to pod:
oc cp /local/path my-pod-name:/container/path

Create tunnel:
oc port-forward my-pod-name local-port:remote-port

The --loglevel level option displays OpenShift API requests, starting with level 6. As you increase the level, up to 10, more information about those requests is added, such as their HTTP request headers and response bodies. Level 10 also includes a curl command to replicate each request.

You can try these two commands, from any project, and compare their outputs. oc get pod --loglevel 10

Oc logs have the --tail option ex:
oc adm node-logs --tail 1 -u kubelet master01