ubuntu The CSI driver must be installed using its Operator - cniackz/public GitHub Wiki
-
Created healthy OpenShift Cluster in the datacenter
-
Install
OpenShift Virtualization
viaOperatorHub
-
Installed DirectPV Helm Operator with OLM:
- Un-install any previous DirectPV:
# 1. Uninstall Operator from OperatorHub in the UI
# 2. Remove all CustomResourceDefinitions and its finalizers from the UI
kubectl directpv uninstall
oc delete catalogsources.operators.coreos.com minio-directpv-operator-catalog
- By finalizer I mean this:
finalizers:
- customresourcecleanup.apiextensions.k8s.io
- From: CRDs:
![Screenshot 2024-04-10 at 9 46 14 AM](https://private-user-images.githubusercontent.com/6667358/321261026-cf7105cb-f7fc-4e36-a922-5ac8b1852393.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUxNzE5OTEsIm5iZiI6MTcyNTE3MTY5MSwicGF0aCI6Ii82NjY3MzU4LzMyMTI2MTAyNi1jZjcxMDVjYi1mN2ZjLTRlMzYtYTkyMi01YWM4YjE4NTIzOTMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkwMSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MDFUMDYyMTMxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9MjIwYjA5YjZiNWNlMGEzMjQ2OTdlOWUwNzU3MTNjYmFiZmIxYzU1ODQ1ZGNmMzMwNjMwMjI3ZWY2ZTk0YTI3NCZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.5R4e44EJoz_Ha5ROSZieXAMZZZOzNyojCcF5AHQPvIE)
- Until all CRDs are gone:
![Screenshot 2024-04-10 at 9 46 59 AM](https://private-user-images.githubusercontent.com/6667358/321261278-886938a0-b047-45f3-809e-4ebec2c30cbe.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUxNzE5OTEsIm5iZiI6MTcyNTE3MTY5MSwicGF0aCI6Ii82NjY3MzU4LzMyMTI2MTI3OC04ODY5MzhhMC1iMDQ3LTQ1ZjMtODA5ZS00ZWJlYzJjMzBjYmUucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkwMSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MDFUMDYyMTMxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9YjI2M2M4OTA5YzA3OWNiYzgwZWQzNDk2NTg2YmFhNmMyOTE4OWYzNGJmNDJjNzQ4Yjg3NTE5M2JkZWJhYjI4NiZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.4QF9m4mN_KUJf2BJMWrLcyukEI7GPfO5yOpP5LFflgQ)
Cesars-MacBook-Pro:~ cniackz$ oc login --token=sha256~P3MVPR084kvEj2MhbeU7H9Ey1nX1MtcA8ebnu_huYac --server=https://api.lab.min.dev:6443 --insecure-skip-tls-verify=true
WARNING: Using insecure TLS client config. Setting this option is not supported!
Logged into "https://api.lab.min.dev:6443" as "kube:admin" using the token provided.
You have access to 77 projects, the list has been suppressed. You can list all projects with 'oc projects'
Using project "default".
Cesars-MacBook-Pro:~ cniackz$
Cesars-MacBook-Pro:~ cniackz$
Cesars-MacBook-Pro:~ cniackz$
Cesars-MacBook-Pro:~ cniackz$
Cesars-MacBook-Pro:~ cniackz$ kubectl directpv uninstall
DirectPV uninstalled successfully
Cesars-MacBook-Pro:~ cniackz$ oc delete catalogsources.operators.coreos.com minio-directpv-operator-catalog
catalogsource.operators.coreos.com "minio-directpv-operator-catalog" deleted
- Install New Operator:
<oc-login> --insecure-skip-tls-verify=true
operator-sdk run bundle quay.io/minio/directpv-operator-bundle:v4.0.10
Expected is:
$ operator-sdk run bundle quay.io/minio/directpv-operator-bundle:v4.0.10
INFO[0014] Creating a File-Based Catalog of the bundle "quay.io/minio/directpv-operator-bundle:v4.0.10"
INFO[0015] Generated a valid File-Based Catalog
INFO[0021] Created registry pod: quay-io-minio-directpv-operator-bundle-v4-0-10
INFO[0022] Created CatalogSource: minio-directpv-operator-catalog
INFO[0022] Created Subscription: minio-directpv-operator-v4-0-10-sub
INFO[0051] Approved InstallPlan install-twgx6 for the Subscription: minio-directpv-operator-v4-0-10-sub
INFO[0051] Waiting for ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" to reach 'Succeeded' phase
INFO[0051] Waiting for ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" to appear
INFO[0054] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Pending
INFO[0056] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: InstallReady
INFO[0057] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Installing
INFO[0077] Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Succeeded
INFO[0077] OLM has successfully installed "minio-directpv-operator.v4.0.10"
- Deploy DirectPV by using CRD of Operator.
File:
https://github.com/minio/directpv/blob/master/operator/examples/my-directpv.yaml
kubectl apply -f - <<EOF
apiVersion: charts.quay.io/v1alpha1
kind: DirectPVChart
metadata:
name: directpvchart-sample
spec:
# Default values copied from <project_dir>/helm-charts/directpv-chart/values.yaml
affinity: {}
fullnameOverride: ""
image:
pullPolicy: IfNotPresent
imagePullSecrets: []
nameOverride: ""
nodeSelector: {}
podAnnotations: {}
podSecurityContext: {}
replicaCount: 1
resources: {}
securityContext: {}
serviceAccount:
annotations: {}
create: true
name: ""
tolerations: []
EOF
- Give permissions (wait 5 minutes for service accounts to appear):
$ oc get serviceaccount -n directpv
NAME SECRETS AGE
builder 1 115s
default 1 115s
deployer 1 115s
directpv-min-io 1 115s
Service Accounts with permissions:
oc adm policy add-scc-to-user privileged -n directpv -z builder
oc adm policy add-scc-to-user privileged -n directpv -z default
oc adm policy add-scc-to-user privileged -n directpv -z deployer
oc adm policy add-scc-to-user privileged -n directpv -z directpv-min-io
- To start with, let's get latest one:
kubectl krew uninstall directpv
kubectl krew install directpv # to move to v4.0.10
Expected is:
$ kubectl directpv --version
directpv version v4.0.10
- Get Information:
kubectl directpv info
When you start fresh, you should see no drives, otherwise release them:
$ kubectl directpv info
┌──────┬──────────┬───────────┬─────────┬────────┐
│ NODE │ CAPACITY │ ALLOCATED │ VOLUMES │ DRIVES │
├──────┼──────────┼───────────┼─────────┼────────┤
│ • 25 │ - │ - │ - │ - │
│ • 26 │ - │ - │ - │ - │
│ • 27 │ - │ - │ - │ - │
└──────┴──────────┴───────────┴─────────┴────────┘
- Discover the drives:
$ kubectl directpv discover
Discovered node '25' ✔
Discovered node '26' ✔
Discovered node '27' ✔
┌─────────────────────┬──────┬─────────┬─────────┬────────────┬──────────────────────────────────────────────┬───────────┬─────────────┐
│ ID │ NODE │ DRIVE │ SIZE │ FILESYSTEM │ MAKE │ AVAILABLE │ DESCRIPTION │
├─────────────────────┼──────┼─────────┼─────────┼────────────┼──────────────────────────────────────────────┼───────────┼─────────────┤
│ 253:4$8SGUcl89Bw... │ 25 │ dm-4 │ 30 GiB │ - │ vg1-35515cc0--9a18--4139--9623--ae44706c3b84 │ YES │ - │
│ 253:5$UfmqoxchS7... │ 25 │ dm-5 │ 30 GiB │ - │ vg1-86e8b558--d625--476a--8086--f6b25118c66a │ YES │ - │
│ 253:6$3V+OSNRXE9... │ 25 │ dm-6 │ 30 GiB │ - │ vg1-8436f57f--65b6--4b83--8053--a1e0fc2e1367 │ YES │ - │
│ 253:7$ULof7Q6HBy... │ 25 │ dm-7 │ 30 GiB │ - │ vg1-3737e692--2bf8--4d5d--851c--2399f3c113c9 │ YES │ - │
│ 253:8$LiwRYdmHgu... │ 25 │ dm-8 │ 30 GiB │ - │ vg1-698b2df0--6c1f--47d9--8210--77263ac55420 │ YES │ - │
│ 253:9$5We8rdbuuh... │ 25 │ dm-9 │ 30 GiB │ - │ vg1-9b0d258f--07e9--4277--b89d--d292b96a72d3 │ YES │ - │
│ 259:2$WICypKjVz5... │ 25 │ nvme0n1 │ 477 GiB │ xfs │ Micron_2200_MTFDHBA512TCK │ YES │ - │
│ 259:1$/hxlH+cPa6... │ 25 │ nvme1n1 │ 932 GiB │ xfs │ INTEL SSDPE2KX010T8 │ YES │ - │
│ 259:0$Yt4EAL8oH+... │ 25 │ nvme2n1 │ 932 GiB │ xfs │ INTEL SSDPE2KX010T8 │ YES │ - │
│ 8:0$MsnrwWf3oXg7... │ 25 │ sda │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:16$U21HicAzo+o... │ 25 │ sdb │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:32$J/Kg2ReEwXw... │ 25 │ sdc │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:80$kvRsZlW11Eg... │ 25 │ sdf │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 253:4$HlFLPmNcFJ... │ 26 │ dm-4 │ 30 GiB │ - │ vg1-bc91fcac--ff01--4e8a--a21d--7a6518a41683 │ YES │ - │
│ 253:5$8XMOjOFX5I... │ 26 │ dm-5 │ 30 GiB │ - │ vg1-7069d247--9f2e--4084--9286--e7f99f3541c4 │ YES │ - │
│ 253:6$gvJMdNF6Pj... │ 26 │ dm-6 │ 30 GiB │ - │ vg1-17c94729--95e7--477b--a8d1--b1ae4d22c32e │ YES │ - │
│ 253:7$N/zEgxyI4o... │ 26 │ dm-7 │ 30 GiB │ - │ vg1-2ca4d503--b7ba--42c2--8598--831af2695b45 │ YES │ - │
│ 253:8$X//wrYhLsW... │ 26 │ dm-8 │ 30 GiB │ - │ vg1-e13c99e8--29ab--4169--94d1--333a2954aee1 │ YES │ - │
│ 253:9$4cqbpzkvkC... │ 26 │ dm-9 │ 30 GiB │ - │ vg1-e3f3efe3--496d--47e4--90ae--8985af17057d │ YES │ - │
│ 259:1$DTzVEjY3iN... │ 26 │ nvme1n1 │ 932 GiB │ xfs │ INTEL SSDPE2KX010T8 │ YES │ - │
│ 259:0$J526GwI7NU... │ 26 │ nvme2n1 │ 932 GiB │ xfs │ INTEL SSDPE2KX010T8 │ YES │ - │
│ 8:0$Ud3dsAjKBw6k... │ 26 │ sda │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:16$RGVCSi6lWEI... │ 26 │ sdb │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:32$DhbndXf7ik3... │ 26 │ sdc │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:48$5xCEhK+neFd... │ 26 │ sdd │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:80$rIXWxOWWJLP... │ 26 │ sdf │ 1.5 TiB │ xfs │ SEAGATE XS1600LE70084 │ YES │ - │
│ 259:2$CZKAGmrWWb... │ 27 │ nvme0n1 │ 477 GiB │ - │ Micron_2200_MTFDHBA512TCK │ YES │ - │
│ 259:0$54AnSdeOYx... │ 27 │ nvme1n1 │ 932 GiB │ xfs │ INTEL SSDPE2KX010T8 │ YES │ - │
│ 259:1$YVZqa9VTUR... │ 27 │ nvme2n1 │ 932 GiB │ xfs │ INTEL SSDPE2KX010T8 │ YES │ - │
│ 8:16$QR040u17fki... │ 27 │ sdb │ 1.5 TiB │ - │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:32$RvjUEhRit6J... │ 27 │ sdc │ 1.5 TiB │ - │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:48$lIL/Xf6r9MK... │ 27 │ sdd │ 1.5 TiB │ - │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:64$ozmiNslceO0... │ 27 │ sde │ 1.5 TiB │ - │ SEAGATE XS1600LE70084 │ YES │ - │
│ 8:80$ReMwg84rAmH... │ 27 │ sdf │ 1.5 TiB │ - │ SEAGATE XS1600LE70084 │ YES │ - │
└─────────────────────┴──────┴─────────┴─────────┴────────────┴──────────────────────────────────────────────┴───────────┴─────────────┘
Generated 'drives.yaml' successfully.
- Init for the first time:
$ kubectl directpv init drives.yaml --dangerous
███████████████████████████████████████████████████████████████████████████ 100%
Processed initialization request '28ca2d59-fc35-45d8-b396-6b7a828a6fcc' for node '27' ✔
Processed initialization request '4f0acf5e-4394-4933-aa8d-be0b1b3d3836' for node '25' ✔
Processed initialization request 'c3c37f5f-03ff-4a1d-b56b-1f0182d71e44' for node '26' ✔
┌──────────────────────────────────────┬──────┬─────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ REQUEST_ID │ NODE │ DRIVE │ MESSAGE │
├──────────────────────────────────────┼──────┼─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ dm-4 │ Failed; unable to format device /dev/dm-4; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=35a26ee7-1447-4e10-b000-527fcfbc62cf -L DIRECTPV /dev/dm-4]; output=mkfs.xfs: /dev/dm-4 appears to contain a partition table (gpt). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ dm-4 │ Failed; unable to format device /dev/dm-4; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=75fa9a3f-b620-493a-ada5-ccfe40f38f58 -L DIRECTPV /dev/dm-4]; output=mkfs.xfs: /dev/dm-4 appears to contain a partition table (gpt). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ dm-5 │ Failed; unable to format device /dev/dm-5; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=9662a346-1989-4107-b7f2-59fb1e59adbf -L DIRECTPV /dev/dm-5]; output=mkfs.xfs: /dev/dm-5 appears to contain a partition table (gpt). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ dm-5 │ Failed; unable to format device /dev/dm-5; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=f9ea26c5-bef3-4056-957e-0132f0b077e2 -L DIRECTPV /dev/dm-5]; output=mkfs.xfs: /dev/dm-5 appears to contain a partition table (gpt). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ dm-6 │ Failed; unable to format device /dev/dm-6; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=4abb5660-f6ee-40e1-ad7d-528bb4027430 -L DIRECTPV /dev/dm-6]; output=mkfs.xfs: /dev/dm-6 appears to contain a partition table (dos). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ dm-6 │ Failed; unable to format device /dev/dm-6; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=576b0110-071e-41c2-9f38-ffd9f1d8440b -L DIRECTPV /dev/dm-6]; output=mkfs.xfs: /dev/dm-6 appears to contain a partition table (dos). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ dm-7 │ Failed; unable to format device /dev/dm-7; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=353734c4-56bc-4bb3-8681-925c08a23a68 -L DIRECTPV /dev/dm-7]; output=mkfs.xfs: /dev/dm-7 appears to contain a partition table (dos). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ dm-7 │ Failed; unable to format device /dev/dm-7; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=3c662660-fa83-496c-92f1-3477f449056c -L DIRECTPV /dev/dm-7]; output=mkfs.xfs: /dev/dm-7 appears to contain a partition table (dos). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ dm-8 │ Failed; unable to format device /dev/dm-8; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=2b0a9932-4aa9-446e-906d-b090883c7ea7 -L DIRECTPV /dev/dm-8]; output=mkfs.xfs: /dev/dm-8 appears to contain a partition table (gpt). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ dm-8 │ Failed; unable to format device /dev/dm-8; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=89e0a06c-59aa-4f88-870b-1570ba3bbc6e -L DIRECTPV /dev/dm-8]; output=mkfs.xfs: /dev/dm-8 appears to contain a partition table (dos). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ dm-9 │ Failed; unable to format device /dev/dm-9; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=9b9ae36d-a1e6-43e3-952f-4711720ee8fd -L DIRECTPV /dev/dm-9]; output=mkfs.xfs: /dev/dm-9 appears to contain a partition table (dos). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ dm-9 │ Failed; unable to format device /dev/dm-9; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=a9aef2a4-37aa-4537-be05-d7021c25a1f3 -L DIRECTPV /dev/dm-9]; output=mkfs.xfs: /dev/dm-9 appears to contain a partition table (gpt). │
│ │ │ │ mkfs.xfs: Use the -f option to force overwrite. │
│ │ │ │ ; error=exit status 1 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ nvme0n1 │ Success │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ nvme1n1 │ Success │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ nvme2n1 │ Success │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ sda │ Success │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ sdb │ Success │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ sdc │ Success │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25 │ sdf │ Success │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ nvme1n1 │ Success │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ nvme2n1 │ Success │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ sda │ Success │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ sdb │ Success │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ sdc │ Success │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ sdd │ Success │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26 │ sdf │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ nvme0n1 │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ nvme1n1 │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ nvme2n1 │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ sdb │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ sdc │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ sdd │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ sde │ Success │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27 │ sdf │ Success │
└──────────────────────────────────────┴──────┴─────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Cesars-MacBook-Pro:~ cniackz$
- Get Info after 1st init:
$ kubectl directpv info
┌──────┬──────────┬───────────┬─────────┬────────┐
│ NODE │ CAPACITY │ ALLOCATED │ VOLUMES │ DRIVES │
├──────┼──────────┼───────────┼─────────┼────────┤
│ • 25 │ 8.1 TiB │ 0 B │ 0 │ 7 │
│ • 26 │ 9.1 TiB │ 0 B │ 0 │ 7 │
│ • 27 │ 9.6 TiB │ 0 B │ 0 │ 8 │
└──────┴──────────┴───────────┴─────────┴────────┘
0 B/27 TiB used, 0 volumes, 22 drives
- Install Operator
- Put a Tenant
- See that DirectPV can be used to get volumes
- Create the access file (run this command from RHEL Host):
- kubeconfig.yaml
64.71.151.82 api.lab.min.dev
64.71.151.83 oauth-openshift.apps.lab.min.dev
64.71.151.83 console-openshift-console.apps.lab.min.dev
64.71.151.83 grafana-openshift-monitoring.apps.lab.min.dev
64.71.151.83 thanos-querier-openshift-monitoring.apps.lab.min.dev
64.71.151.83 prometheus-k8s-openshift-monitoring.apps.lab.min.dev
64.71.151.83 alertmanager-main-openshift-monitoring.apps.lab.min.dev
cd ~
oc login --token=sha256~xSxMwTNUal_NGAeNtn5MhCJMF8WunwK9eYgO4BzhwNI --server=https://api.lab.min.dev:6443 --insecure-skip-tls-verify=true
oc config view --raw > kubeconfig.yaml
- Get these files:
https://github.com/Praveenrajmani/redhat-certification <--- Over this repo I don't have control, just document it!
- Run test from run tests
podman login registry.redhat.io
podman run --privileged -v `pwd`:/data --rm -it registry.redhat.io/openshift4/ose-tests sh -c "KUBECONFIG=/data/kubeconfig.yaml TEST_CSI_DRIVER_FILES=/data/manifest.yaml /usr/bin/openshift-tests run openshift/csi --junit-dir /data/results"
- Thu Jan 11 2024 @ 11:41 AM GDL Time executing with version
4.0.10
Suite run returned error: 5 fail, 36 pass, 213 skip (6m53s)
Storage Capabilities (guaranteed only on full CSI test suite with 0 fails)
==========================================================================
Driver short name: directpv
Driver name: directpv.min.io
Storage class: storageclass.yaml
Supported OpenShift / CSI features:
Persistent volumes: true
Raw block mode: false
FSGroup: true
Executable files on a volume: true
Volume snapshots: false
Volume cloning: false
Use volume from multiple pods on a node: false
ReadWriteMany access mode: false
Volume expansion for controller: false
Volume expansion for node: true
Volume limits: false
Volume can run on single node: true
Topology: true
Supported OpenShift Virtualization features:
Raw block VM disks: false
Live migration: false
VM snapshots: false
Storage-assisted cloning: false
error running options: 5 fail, 36 pass, 213 skip (6m53s)error: 5 fail, 36 pass, 213 skip (6m53s)
- Mon Feb 26 2024 @ 7 PM EST Time executing with version
4.0.10
Suite run returned error: 34 fail, 7 pass, 213 skip (32m46s)
Storage Capabilities (guaranteed only on full CSI test suite with 0 fails)
==========================================================================
Driver short name: directpv
Driver name: directpv.min.io
Storage class: storageclass.yaml
Supported OpenShift / CSI features:
Persistent volumes: true
Raw block mode: false
FSGroup: true
Executable files on a volume: true
Volume snapshots: false
Volume cloning: false
Use volume from multiple pods on a node: false
ReadWriteMany access mode: false
Volume expansion for controller: false
Volume expansion for node: true
Volume limits: false
Volume can run on single node: true
Topology: true
Supported OpenShift Virtualization features:
Raw block VM disks: false
Live migration: false
VM snapshots: false
Storage-assisted cloning: false
error running options: 34 fail, 7 pass, 213 skip (32m46s)error: 34 fail, 7 pass, 213 skip (32m46s)
- Mon Apr 8 2024 9:32 pm EST executing with version
4.0.10
Suite run returned error: 1 fail, 40 pass, 213 skip (6m27s)
Storage Capabilities (guaranteed only on full CSI test suite with 0 fails)
==========================================================================
Driver short name: directpv
Driver name: [directpv.min.io](http://directpv.min.io/)
Storage class: storageclass.yaml
Supported OpenShift / CSI features:
Persistent volumes: true
Raw block mode: false
FSGroup: true
Executable files on a volume: true
Volume snapshots: false
Volume cloning: false
Use volume from multiple pods on a node: false
ReadWriteMany access mode: false
Volume expansion for controller: false
Volume expansion for node: true
Volume limits: false
Volume can run on single node: true
Topology: true
Supported OpenShift Virtualization features:
Raw block VM disks: false
Live migration: false
VM snapshots: false
Storage-assisted cloning: false
error running options: 1 fail, 40 pass, 213 skip (6m27s)error: 1 fail, 40 pass, 213 skip (6m27s)
tar czf results_8.tar.gz results
- https://github.com/cniackz/directpvcertification/blob/main/results-6.tar.gz
- https://github.com/cniackz/directpvcertification/blob/main/results-7.tar.gz
- https://github.com/cniackz/directpvcertification/blob/main/results-8.tar.gz
- After executing the CSI Tests, we observed this issue on April 9th, could it be because of DirectPV Helm Operator:
![Screenshot 2024-04-09 at 2 30 17 PM](https://private-user-images.githubusercontent.com/6667358/320964271-7662f2fc-9d39-4f09-8ca1-02be33494413.png?jwt=eyJhbGciOiJIUzI1NiIsInR5cCI6IkpXVCJ9.eyJpc3MiOiJnaXRodWIuY29tIiwiYXVkIjoicmF3LmdpdGh1YnVzZXJjb250ZW50LmNvbSIsImtleSI6ImtleTUiLCJleHAiOjE3MjUxNzE5OTEsIm5iZiI6MTcyNTE3MTY5MSwicGF0aCI6Ii82NjY3MzU4LzMyMDk2NDI3MS03NjYyZjJmYy05ZDM5LTRmMDktOGNhMS0wMmJlMzM0OTQ0MTMucG5nP1gtQW16LUFsZ29yaXRobT1BV1M0LUhNQUMtU0hBMjU2JlgtQW16LUNyZWRlbnRpYWw9QUtJQVZDT0RZTFNBNTNQUUs0WkElMkYyMDI0MDkwMSUyRnVzLWVhc3QtMSUyRnMzJTJGYXdzNF9yZXF1ZXN0JlgtQW16LURhdGU9MjAyNDA5MDFUMDYyMTMxWiZYLUFtei1FeHBpcmVzPTMwMCZYLUFtei1TaWduYXR1cmU9ZGUwMzI5MWI4ZTVhOWVmYjhhYTEwNGUwM2RmNTU0OWM4M2VjMzg2M2JkODJlMmE3ZjA3ZDE0MmRlYjdjOTRkYSZYLUFtei1TaWduZWRIZWFkZXJzPWhvc3QmYWN0b3JfaWQ9MCZrZXlfaWQ9MCZyZXBvX2lkPTAifQ.b3x11hs5VwGd9qRWW-d3nThlePCFdrtjX745iJRNJgE)