Cluster&APP Health - Almax84/consulting-wiki GitHub Wiki
oc get nodes
Displays a column with the status of each node. If a node is not Ready, then it cannot communicate with the OpenShift control plane, and is effectively dead to the cluster.
oc adm top nodes
Displays the current CPU and memory usage of each node. These are actual usage numbers, not the resource requests that the OpenShift scheduler considers as the available and used capacity of the node.
oc describe node my-node-name
Displays the resources available and used from the scheduler point of view, and other information. Look for the headings "Capacity," "Allocatable," and "Allocated resources" in the output. The heading "Conditions" indicates whether the node is under memory pressure, disk pressure, or some other condition that would prevent the node from starting new containers.
oc get clusterversion
The output lists the version, including minor releases, the cluster uptime for a given version, and the overall status of the cluster.
oc describe clusterversion
More detailed output
oc get clusteroperators
to retrieve the list of all cluster operators
DISPLAYING THE LOGS OF A NODE
An OpenShift node based on Red Hat Enterprise Linux CoreOS runs very few local services that would require direct access to a node to inspect their status. Most of the system services in Red Hat Enterprise Linux CoreOS run as containers. The main exceptions are the CRI-O container engine and the Kubelet, which are Systemd units. To view these logs, use the oc adm node-logs command as shown in the following examples:
[user@host ~]$ oc adm node-logs -u crio my-node-name
[user@host ~]$ oc adm node-logs -u kubelet my-node-name
You can also display all journal logs of a node:
[user@host ~]$ oc adm node-logs my-node-name
Opening a Shell Prompt on an OpenShift Node
[user@host ~]$ oc debug node/my-node-name ...output omitted... sh-4.4# chroot /host sh-4.4# systemctl is-active kubelet active
APPLICATION LOGS & DEBUG
oc logs my-pod-name
If the pods contain multiple containers
oc logs my-pod-name -c my-container-name
Create troubleshooting pods:
oc debug deployment/my-deployment-name --as-root
Connect to the pod: oc rsh my-pod-name
Copy file from/to pod:
oc cp /local/path my-pod-name:/container/path
Create tunnel:
oc port-forward my-pod-name local-port:remote-port
The --loglevel level option displays OpenShift API requests, starting with level 6. As you increase the level, up to 10, more information about those requests is added, such as their HTTP request headers and response bodies. Level 10 also includes a curl command to replicate each request.
You can try these two commands, from any project, and compare their outputs. oc get pod --loglevel 10
Oc logs have the --tail option ex:
oc adm node-logs --tail 1 -u kubelet master01