ubuntu The CSI driver must be installed using its Operator - cniackz/public GitHub Wiki

Steps:

  1. Created healthy OpenShift Cluster in the datacenter

  2. Install OpenShift Virtualization via OperatorHub

  3. Installed DirectPV Helm Operator with OLM:

  • Un-install any previous DirectPV:
# 1. Uninstall Operator from OperatorHub in the UI
# 2. Remove all CustomResourceDefinitions and its finalizers from the UI
kubectl directpv uninstall
oc delete catalogsources.operators.coreos.com minio-directpv-operator-catalog
  • By finalizer I mean this:
  finalizers:
    - customresourcecleanup.apiextensions.k8s.io
  • From: CRDs:
Screenshot 2024-04-10 at 9 46 14 AM
  • Until all CRDs are gone:
Screenshot 2024-04-10 at 9 46 59 AM
Cesars-MacBook-Pro:~ cniackz$ oc login --token=sha256~P3MVPR084kvEj2MhbeU7H9Ey1nX1MtcA8ebnu_huYac --server=https://api.lab.min.dev:6443 --insecure-skip-tls-verify=true
WARNING: Using insecure TLS client config. Setting this option is not supported!

Logged into "https://api.lab.min.dev:6443" as "kube:admin" using the token provided.

You have access to 77 projects, the list has been suppressed. You can list all projects with 'oc projects'

Using project "default".
Cesars-MacBook-Pro:~ cniackz$ 
Cesars-MacBook-Pro:~ cniackz$ 
Cesars-MacBook-Pro:~ cniackz$ 
Cesars-MacBook-Pro:~ cniackz$ 
Cesars-MacBook-Pro:~ cniackz$ kubectl directpv uninstall
DirectPV uninstalled successfully
Cesars-MacBook-Pro:~ cniackz$ oc delete catalogsources.operators.coreos.com minio-directpv-operator-catalog
catalogsource.operators.coreos.com "minio-directpv-operator-catalog" deleted
  • Install New Operator:
<oc-login> --insecure-skip-tls-verify=true
operator-sdk run bundle quay.io/minio/directpv-operator-bundle:v4.0.10

Expected is:

$ operator-sdk run bundle quay.io/minio/directpv-operator-bundle:v4.0.10
INFO[0014] Creating a File-Based Catalog of the bundle "quay.io/minio/directpv-operator-bundle:v4.0.10" 
INFO[0015] Generated a valid File-Based Catalog         
INFO[0021] Created registry pod: quay-io-minio-directpv-operator-bundle-v4-0-10 
INFO[0022] Created CatalogSource: minio-directpv-operator-catalog 
INFO[0022] Created Subscription: minio-directpv-operator-v4-0-10-sub 
INFO[0051] Approved InstallPlan install-twgx6 for the Subscription: minio-directpv-operator-v4-0-10-sub 
INFO[0051] Waiting for ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" to reach 'Succeeded' phase 
INFO[0051]   Waiting for ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" to appear 
INFO[0054]   Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Pending 
INFO[0056]   Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: InstallReady 
INFO[0057]   Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Installing 
INFO[0077]   Found ClusterServiceVersion "default/minio-directpv-operator.v4.0.10" phase: Succeeded 
INFO[0077] OLM has successfully installed "minio-directpv-operator.v4.0.10"
  1. Deploy DirectPV by using CRD of Operator.

File: https://github.com/minio/directpv/blob/master/operator/examples/my-directpv.yaml

kubectl apply -f - <<EOF
apiVersion: charts.quay.io/v1alpha1
kind: DirectPVChart
metadata:
  name: directpvchart-sample
spec:
  # Default values copied from <project_dir>/helm-charts/directpv-chart/values.yaml
  affinity: {}
  fullnameOverride: ""
  image:
    pullPolicy: IfNotPresent
  imagePullSecrets: []
  nameOverride: ""
  nodeSelector: {}
  podAnnotations: {}
  podSecurityContext: {}
  replicaCount: 1
  resources: {}
  securityContext: {}
  serviceAccount:
    annotations: {}
    create: true
    name: ""
  tolerations: []
EOF
  1. Give permissions (wait 5 minutes for service accounts to appear):
$ oc get serviceaccount -n directpv
NAME              SECRETS   AGE
builder           1         115s
default           1         115s
deployer          1         115s
directpv-min-io   1         115s

Service Accounts with permissions:

oc adm policy add-scc-to-user privileged -n directpv -z builder
oc adm policy add-scc-to-user privileged -n directpv -z default
oc adm policy add-scc-to-user privileged -n directpv -z deployer
oc adm policy add-scc-to-user privileged -n directpv -z directpv-min-io

Using krew plugin for DirectPV:

  1. To start with, let's get latest one:
kubectl krew uninstall directpv
kubectl krew install directpv # to move to v4.0.10

Expected is:

$ kubectl directpv --version
directpv version v4.0.10
  1. Get Information:
kubectl directpv info

When you start fresh, you should see no drives, otherwise release them:

$ kubectl directpv info
┌──────┬──────────┬───────────┬─────────┬────────┐
│ NODE │ CAPACITY │ ALLOCATED │ VOLUMES │ DRIVES │
├──────┼──────────┼───────────┼─────────┼────────┤
│ • 25 │ -        │ -         │ -       │ -      │
│ • 26 │ -        │ -         │ -       │ -      │
│ • 27 │ -        │ -         │ -       │ -      │
└──────┴──────────┴───────────┴─────────┴────────┘
  1. Discover the drives:
$ kubectl directpv discover

 Discovered node '25' ✔
 Discovered node '26' ✔
 Discovered node '27' ✔

┌─────────────────────┬──────┬─────────┬─────────┬────────────┬──────────────────────────────────────────────┬───────────┬─────────────┐
│ ID                  │ NODE │ DRIVE   │ SIZE    │ FILESYSTEM │ MAKE                                         │ AVAILABLE │ DESCRIPTION │
├─────────────────────┼──────┼─────────┼─────────┼────────────┼──────────────────────────────────────────────┼───────────┼─────────────┤
│ 253:4$8SGUcl89Bw... │ 25   │ dm-4    │ 30 GiB  │ -          │ vg1-35515cc0--9a18--4139--9623--ae44706c3b84 │ YES       │ -           │
│ 253:5$UfmqoxchS7... │ 25   │ dm-5    │ 30 GiB  │ -          │ vg1-86e8b558--d625--476a--8086--f6b25118c66a │ YES       │ -           │
│ 253:6$3V+OSNRXE9... │ 25   │ dm-6    │ 30 GiB  │ -          │ vg1-8436f57f--65b6--4b83--8053--a1e0fc2e1367 │ YES       │ -           │
│ 253:7$ULof7Q6HBy... │ 25   │ dm-7    │ 30 GiB  │ -          │ vg1-3737e692--2bf8--4d5d--851c--2399f3c113c9 │ YES       │ -           │
│ 253:8$LiwRYdmHgu... │ 25   │ dm-8    │ 30 GiB  │ -          │ vg1-698b2df0--6c1f--47d9--8210--77263ac55420 │ YES       │ -           │
│ 253:9$5We8rdbuuh... │ 25   │ dm-9    │ 30 GiB  │ -          │ vg1-9b0d258f--07e9--4277--b89d--d292b96a72d3 │ YES       │ -           │
│ 259:2$WICypKjVz5... │ 25   │ nvme0n1 │ 477 GiB │ xfs        │ Micron_2200_MTFDHBA512TCK                    │ YES       │ -           │
│ 259:1$/hxlH+cPa6... │ 25   │ nvme1n1 │ 932 GiB │ xfs        │ INTEL SSDPE2KX010T8                          │ YES       │ -           │
│ 259:0$Yt4EAL8oH+... │ 25   │ nvme2n1 │ 932 GiB │ xfs        │ INTEL SSDPE2KX010T8                          │ YES       │ -           │
│ 8:0$MsnrwWf3oXg7... │ 25   │ sda     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:16$U21HicAzo+o... │ 25   │ sdb     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:32$J/Kg2ReEwXw... │ 25   │ sdc     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:80$kvRsZlW11Eg... │ 25   │ sdf     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 253:4$HlFLPmNcFJ... │ 26   │ dm-4    │ 30 GiB  │ -          │ vg1-bc91fcac--ff01--4e8a--a21d--7a6518a41683 │ YES       │ -           │
│ 253:5$8XMOjOFX5I... │ 26   │ dm-5    │ 30 GiB  │ -          │ vg1-7069d247--9f2e--4084--9286--e7f99f3541c4 │ YES       │ -           │
│ 253:6$gvJMdNF6Pj... │ 26   │ dm-6    │ 30 GiB  │ -          │ vg1-17c94729--95e7--477b--a8d1--b1ae4d22c32e │ YES       │ -           │
│ 253:7$N/zEgxyI4o... │ 26   │ dm-7    │ 30 GiB  │ -          │ vg1-2ca4d503--b7ba--42c2--8598--831af2695b45 │ YES       │ -           │
│ 253:8$X//wrYhLsW... │ 26   │ dm-8    │ 30 GiB  │ -          │ vg1-e13c99e8--29ab--4169--94d1--333a2954aee1 │ YES       │ -           │
│ 253:9$4cqbpzkvkC... │ 26   │ dm-9    │ 30 GiB  │ -          │ vg1-e3f3efe3--496d--47e4--90ae--8985af17057d │ YES       │ -           │
│ 259:1$DTzVEjY3iN... │ 26   │ nvme1n1 │ 932 GiB │ xfs        │ INTEL SSDPE2KX010T8                          │ YES       │ -           │
│ 259:0$J526GwI7NU... │ 26   │ nvme2n1 │ 932 GiB │ xfs        │ INTEL SSDPE2KX010T8                          │ YES       │ -           │
│ 8:0$Ud3dsAjKBw6k... │ 26   │ sda     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:16$RGVCSi6lWEI... │ 26   │ sdb     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:32$DhbndXf7ik3... │ 26   │ sdc     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:48$5xCEhK+neFd... │ 26   │ sdd     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:80$rIXWxOWWJLP... │ 26   │ sdf     │ 1.5 TiB │ xfs        │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 259:2$CZKAGmrWWb... │ 27   │ nvme0n1 │ 477 GiB │ -          │ Micron_2200_MTFDHBA512TCK                    │ YES       │ -           │
│ 259:0$54AnSdeOYx... │ 27   │ nvme1n1 │ 932 GiB │ xfs        │ INTEL SSDPE2KX010T8                          │ YES       │ -           │
│ 259:1$YVZqa9VTUR... │ 27   │ nvme2n1 │ 932 GiB │ xfs        │ INTEL SSDPE2KX010T8                          │ YES       │ -           │
│ 8:16$QR040u17fki... │ 27   │ sdb     │ 1.5 TiB │ -          │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:32$RvjUEhRit6J... │ 27   │ sdc     │ 1.5 TiB │ -          │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:48$lIL/Xf6r9MK... │ 27   │ sdd     │ 1.5 TiB │ -          │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:64$ozmiNslceO0... │ 27   │ sde     │ 1.5 TiB │ -          │ SEAGATE XS1600LE70084                        │ YES       │ -           │
│ 8:80$ReMwg84rAmH... │ 27   │ sdf     │ 1.5 TiB │ -          │ SEAGATE XS1600LE70084                        │ YES       │ -           │
└─────────────────────┴──────┴─────────┴─────────┴────────────┴──────────────────────────────────────────────┴───────────┴─────────────┘

Generated 'drives.yaml' successfully.
  1. Init for the first time:
$ kubectl directpv init drives.yaml --dangerous

 ███████████████████████████████████████████████████████████████████████████ 100%

 Processed initialization request '28ca2d59-fc35-45d8-b396-6b7a828a6fcc' for node '27' ✔
 Processed initialization request '4f0acf5e-4394-4933-aa8d-be0b1b3d3836' for node '25' ✔
 Processed initialization request 'c3c37f5f-03ff-4a1d-b56b-1f0182d71e44' for node '26' ✔

┌──────────────────────────────────────┬──────┬─────────┬─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│ REQUEST_ID                           │ NODE │ DRIVE   │ MESSAGE                                                                                                                                                                                                                                 │
├──────────────────────────────────────┼──────┼─────────┼─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┤
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ dm-4    │ Failed; unable to format device /dev/dm-4; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=35a26ee7-1447-4e10-b000-527fcfbc62cf -L DIRECTPV /dev/dm-4]; output=mkfs.xfs: /dev/dm-4 appears to contain a partition table (gpt). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ dm-4    │ Failed; unable to format device /dev/dm-4; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=75fa9a3f-b620-493a-ada5-ccfe40f38f58 -L DIRECTPV /dev/dm-4]; output=mkfs.xfs: /dev/dm-4 appears to contain a partition table (gpt). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ dm-5    │ Failed; unable to format device /dev/dm-5; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=9662a346-1989-4107-b7f2-59fb1e59adbf -L DIRECTPV /dev/dm-5]; output=mkfs.xfs: /dev/dm-5 appears to contain a partition table (gpt). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ dm-5    │ Failed; unable to format device /dev/dm-5; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=f9ea26c5-bef3-4056-957e-0132f0b077e2 -L DIRECTPV /dev/dm-5]; output=mkfs.xfs: /dev/dm-5 appears to contain a partition table (gpt). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ dm-6    │ Failed; unable to format device /dev/dm-6; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=4abb5660-f6ee-40e1-ad7d-528bb4027430 -L DIRECTPV /dev/dm-6]; output=mkfs.xfs: /dev/dm-6 appears to contain a partition table (dos). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ dm-6    │ Failed; unable to format device /dev/dm-6; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=576b0110-071e-41c2-9f38-ffd9f1d8440b -L DIRECTPV /dev/dm-6]; output=mkfs.xfs: /dev/dm-6 appears to contain a partition table (dos). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ dm-7    │ Failed; unable to format device /dev/dm-7; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=353734c4-56bc-4bb3-8681-925c08a23a68 -L DIRECTPV /dev/dm-7]; output=mkfs.xfs: /dev/dm-7 appears to contain a partition table (dos). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ dm-7    │ Failed; unable to format device /dev/dm-7; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=3c662660-fa83-496c-92f1-3477f449056c -L DIRECTPV /dev/dm-7]; output=mkfs.xfs: /dev/dm-7 appears to contain a partition table (dos). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ dm-8    │ Failed; unable to format device /dev/dm-8; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=2b0a9932-4aa9-446e-906d-b090883c7ea7 -L DIRECTPV /dev/dm-8]; output=mkfs.xfs: /dev/dm-8 appears to contain a partition table (gpt). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ dm-8    │ Failed; unable to format device /dev/dm-8; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=89e0a06c-59aa-4f88-870b-1570ba3bbc6e -L DIRECTPV /dev/dm-8]; output=mkfs.xfs: /dev/dm-8 appears to contain a partition table (dos). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ dm-9    │ Failed; unable to format device /dev/dm-9; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=9b9ae36d-a1e6-43e3-952f-4711720ee8fd -L DIRECTPV /dev/dm-9]; output=mkfs.xfs: /dev/dm-9 appears to contain a partition table (dos). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ dm-9    │ Failed; unable to format device /dev/dm-9; unable to execute command [mkfs.xfs -i maxpct=50 -m uuid=a9aef2a4-37aa-4537-be05-d7021c25a1f3 -L DIRECTPV /dev/dm-9]; output=mkfs.xfs: /dev/dm-9 appears to contain a partition table (gpt). │
│                                      │      │         │ mkfs.xfs: Use the -f option to force overwrite.                                                                                                                                                                                         │
│                                      │      │         │ ; error=exit status 1                                                                                                                                                                                                                   │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ nvme0n1 │ Success                                                                                                                                                                                                                                 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ nvme1n1 │ Success                                                                                                                                                                                                                                 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ nvme2n1 │ Success                                                                                                                                                                                                                                 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ sda     │ Success                                                                                                                                                                                                                                 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ sdb     │ Success                                                                                                                                                                                                                                 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ sdc     │ Success                                                                                                                                                                                                                                 │
│ 4f0acf5e-4394-4933-aa8d-be0b1b3d3836 │ 25   │ sdf     │ Success                                                                                                                                                                                                                                 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ nvme1n1 │ Success                                                                                                                                                                                                                                 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ nvme2n1 │ Success                                                                                                                                                                                                                                 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ sda     │ Success                                                                                                                                                                                                                                 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ sdb     │ Success                                                                                                                                                                                                                                 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ sdc     │ Success                                                                                                                                                                                                                                 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ sdd     │ Success                                                                                                                                                                                                                                 │
│ c3c37f5f-03ff-4a1d-b56b-1f0182d71e44 │ 26   │ sdf     │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ nvme0n1 │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ nvme1n1 │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ nvme2n1 │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ sdb     │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ sdc     │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ sdd     │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ sde     │ Success                                                                                                                                                                                                                                 │
│ 28ca2d59-fc35-45d8-b396-6b7a828a6fcc │ 27   │ sdf     │ Success                                                                                                                                                                                                                                 │
└──────────────────────────────────────┴──────┴─────────┴─────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────┘
Cesars-MacBook-Pro:~ cniackz$
  1. Get Info after 1st init:
$ kubectl directpv info
┌──────┬──────────┬───────────┬─────────┬────────┐
│ NODE │ CAPACITY │ ALLOCATED │ VOLUMES │ DRIVES │
├──────┼──────────┼───────────┼─────────┼────────┤
│ • 25 │ 8.1 TiB  │ 0 B       │ 0       │ 7      │
│ • 26 │ 9.1 TiB  │ 0 B       │ 0       │ 7      │
│ • 27 │ 9.6 TiB  │ 0 B       │ 0       │ 8      │
└──────┴──────────┴───────────┴─────────┴────────┘

0 B/27 TiB used, 0 volumes, 22 drives

Test DirectPV is working with MinIO

  1. Install Operator
  2. Put a Tenant
  3. See that DirectPV can be used to get volumes

Execute CSI Test:

  1. Create the access file (run this command from RHEL Host):
  • kubeconfig.yaml
64.71.151.82	api.lab.min.dev
64.71.151.83	oauth-openshift.apps.lab.min.dev
64.71.151.83	console-openshift-console.apps.lab.min.dev
64.71.151.83	grafana-openshift-monitoring.apps.lab.min.dev
64.71.151.83	thanos-querier-openshift-monitoring.apps.lab.min.dev
64.71.151.83	prometheus-k8s-openshift-monitoring.apps.lab.min.dev
64.71.151.83	alertmanager-main-openshift-monitoring.apps.lab.min.dev
cd ~

oc login --token=sha256~xSxMwTNUal_NGAeNtn5MhCJMF8WunwK9eYgO4BzhwNI --server=https://api.lab.min.dev:6443 --insecure-skip-tls-verify=true

oc config view --raw > kubeconfig.yaml
  1. Get these files:

https://github.com/Praveenrajmani/redhat-certification <--- Over this repo I don't have control, just document it!

  1. Run test from run tests
podman login registry.redhat.io
podman run --privileged -v `pwd`:/data --rm -it  registry.redhat.io/openshift4/ose-tests sh -c "KUBECONFIG=/data/kubeconfig.yaml TEST_CSI_DRIVER_FILES=/data/manifest.yaml /usr/bin/openshift-tests run openshift/csi --junit-dir /data/results"
  • Thu Jan 11 2024 @ 11:41 AM GDL Time executing with version 4.0.10
Suite run returned error: 5 fail, 36 pass, 213 skip (6m53s)

Storage Capabilities (guaranteed only on full CSI test suite with 0 fails)
==========================================================================
Driver short name:                         directpv
Driver name:                               directpv.min.io
Storage class:                             storageclass.yaml
Supported OpenShift / CSI features:
  Persistent volumes:                      true
  Raw block mode:                          false
  FSGroup:                                 true
  Executable files on a volume:            true
  Volume snapshots:                        false
  Volume cloning:                          false
  Use volume from multiple pods on a node: false
  ReadWriteMany access mode:               false
  Volume expansion for controller:         false
  Volume expansion for node:               true
  Volume limits:                           false
  Volume can run on single node:           true
  Topology:                                true
Supported OpenShift Virtualization features:
  Raw block VM disks:                      false
  Live migration:                          false
  VM snapshots:                            false
  Storage-assisted cloning:                false

error running options: 5 fail, 36 pass, 213 skip (6m53s)error: 5 fail, 36 pass, 213 skip (6m53s)
  • Mon Feb 26 2024 @ 7 PM EST Time executing with version 4.0.10
Suite run returned error: 34 fail, 7 pass, 213 skip (32m46s)

Storage Capabilities (guaranteed only on full CSI test suite with 0 fails)
==========================================================================
Driver short name:                         directpv
Driver name:                               directpv.min.io
Storage class:                             storageclass.yaml
Supported OpenShift / CSI features:
  Persistent volumes:                      true
  Raw block mode:                          false
  FSGroup:                                 true
  Executable files on a volume:            true
  Volume snapshots:                        false
  Volume cloning:                          false
  Use volume from multiple pods on a node: false
  ReadWriteMany access mode:               false
  Volume expansion for controller:         false
  Volume expansion for node:               true
  Volume limits:                           false
  Volume can run on single node:           true
  Topology:                                true
Supported OpenShift Virtualization features:
  Raw block VM disks:                      false
  Live migration:                          false
  VM snapshots:                            false
  Storage-assisted cloning:                false

error running options: 34 fail, 7 pass, 213 skip (32m46s)error: 34 fail, 7 pass, 213 skip (32m46s)

  • Mon Apr 8 2024 9:32 pm EST executing with version 4.0.10
Suite run returned error: 1 fail, 40 pass, 213 skip (6m27s)

Storage Capabilities (guaranteed only on full CSI test suite with 0 fails)
==========================================================================
Driver short name:                         directpv
Driver name:                               [directpv.min.io](http://directpv.min.io/)
Storage class:                             storageclass.yaml
Supported OpenShift / CSI features:
  Persistent volumes:                      true
  Raw block mode:                          false
  FSGroup:                                 true
  Executable files on a volume:            true
  Volume snapshots:                        false
  Volume cloning:                          false
  Use volume from multiple pods on a node: false
  ReadWriteMany access mode:               false
  Volume expansion for controller:         false
  Volume expansion for node:               true
  Volume limits:                           false
  Volume can run on single node:           true
  Topology:                                true
Supported OpenShift Virtualization features:
  Raw block VM disks:                      false
  Live migration:                          false
  VM snapshots:                            false
  Storage-assisted cloning:                false

error running options: 1 fail, 40 pass, 213 skip (6m27s)error: 1 fail, 40 pass, 213 skip (6m27s)

Compress Results:

tar czf results_8.tar.gz results

Save result:

Case:

Issues:

  • After executing the CSI Tests, we observed this issue on April 9th, could it be because of DirectPV Helm Operator:
Screenshot 2024-04-09 at 2 30 17 PM
⚠️ **GitHub.com Fallback** ⚠️