How to update old Operator to new Operator 4.0.9 to newest version - cniackz/public GitHub Wiki

Objective:

  • Migrate from 4.0.9 to latest by not removing the data.
  • DON'T DELETE THE PVCs NOR the PVs, Keep Your Data intact.
    • As long as you keep your data, you can work to update the other resources.

Steps:

  1. Assuming you are coming from a very old version like 4.0.9 of Operator
$ k version
Client Version: version.Info{Major:"1", Minor:"23", GitVersion:"v1.23.0", GitCommit:"ab69524f795c42094a6630298ff53f3c3ebab7f4", GitTreeState:"clean", BuildDate:"2021-12-07T18:16:20Z", GoVersion:"go1.17.3", Compiler:"gc", Platform:"darwin/arm64"}
Server Version: version.Info{Major:"1", Minor:"21", GitVersion:"v1.21.1", GitCommit:"5e58841cce77d4bc13713ad2b91fa0d961e69192", GitTreeState:"clean", BuildDate:"2021-05-21T23:06:30Z", GoVersion:"go1.16.4", Compiler:"gc", Platform:"linux/arm64"}
WARNING: version difference between client (1.23) and server (1.21) exceeds the supported minor version skew of +/-1

Where client version and server version are less than 1.24 in both client and server

  1. My first recommendation is to update kubernetes version in your client and server because 1.23 or 1.21 is too old already and new Operator version is intended to run in newer versions of k8s anyway. So let's do it.

https://kubernetes.io/docs/tasks/administer-cluster/kubeadm/kubeadm-upgrade/

  1. Updated the images of the Operator Deployment, operator and console:
image: minio/operator:v4.5.4
image: minio/console:v0.21.1
  1. Applied changes for new operator:
kubectl apply -k github.com/minio/operator/

$ kubectl apply -k github.com/minio/operator/
namespace/minio-operator unchanged
customresourcedefinition.apiextensions.k8s.io/tenants.minio.min.io configured
serviceaccount/console-sa unchanged
serviceaccount/minio-operator unchanged
clusterrole.rbac.authorization.k8s.io/console-sa-role configured
clusterrole.rbac.authorization.k8s.io/minio-operator-role configured
clusterrolebinding.rbac.authorization.k8s.io/console-sa-binding unchanged
clusterrolebinding.rbac.authorization.k8s.io/minio-operator-binding unchanged
configmap/console-env unchanged
secret/console-sa-secret created
service/console configured
service/operator configured
deployment.apps/console configured
deployment.apps/minio-operator configured
  1. If you see similar message:
Tenants.minio.min.io is forbidden: User "system:serviceaccount:minio-operator:default" cannot list resource "tenants"  in API group "minio.min.io" at the cluster scope

Then do this:

  • File Name: clusterrole.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
  name: default-role-cesar
  namespace: minio-operator
rules:
- apiGroups: ["minio.min.io"]
  resources: ["services"]
  verbs: ["get","watch","list"]
- apiGroups: ["minio.min.io"]
  resources: ["pods"]
  verbs: ["get","watch","list"]
- apiGroups: ["minio.min.io"]
  resources: ["ingresses"]
  verbs: ["get","watch","list"]
- apiGroups: ["minio.min.io"]
  resources: ["nodes"]
  verbs: ["list","watch"]
- apiGroups: ["minio.min.io"]
  resources: ["tenants"]
  verbs: ["get","watch","list"]
  • File: clusterrolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: default-role-cesar-binding
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: default-role-cesar
subjects:
- kind: ServiceAccount
  name: default
  namespace: minio-operator

$ k apply -f clusterrole.yaml
$ k apply -f clusterrolebinding.yaml

Then your new Operator will see the old Tenant:

  1. Update MinIO Image on Tenant Specification:
spec:
  ## Registry location and Tag to download MinIO Server image
  image: quay.io/minio/minio:RELEASE.2022-11-11T03-44-20Z

Command example to edit:

$ k edit tenants cesar -n default
tenant.minio.min.io/cesar edited
  1. If you face permission issue like this:
Error: file access denied (cmd.StorageErr)
      11: internal/logger/logger.go:258:logger.LogIf()
      10: cmd/erasure-object.go:1279:cmd.erasureObjects.putObject()
       9: cmd/erasure-object.go:937:cmd.erasureObjects.PutObject()
       8: cmd/erasure-sets.go:940:cmd.(*erasureSets).PutObject()
       7: cmd/config-common.go:83:cmd.saveConfigWithOpts()
       6: cmd/config-common.go:88:cmd.saveConfig()
       5: cmd/erasure-server-pool-decom.go:493:cmd.poolMeta.save()
       4: cmd/erasure-server-pool-decom.go:586:cmd.(*erasureServerPools).Init()
       3: cmd/erasure-server-pool.go:130:cmd.newErasureServerPools()
       2: cmd/server-main.go:741:cmd.newObjectLayer()
       1: cmd/server-main.go:576:cmd.serverMain()
ERROR Unable to initialize backend: Prefix access is denied: .minio.sys/pool.bin

All you need to do is let cluster pick proper permissions by setting securityContext as empty under pool in tenant config:

pool:
  ## Configure security context
  securityContext: {}

Then wait a bit and you will see:

All MinIO sub-systems initialized successfully in 40.866292ms
MinIO Object Storage Server
Copyright: 2015-2022 MinIO, Inc.
License: GNU AGPLv3 <https://www.gnu.org/licenses/agpl-3.0.html>
Version: RELEASE.2022-11-11T03-44-20Z (go1.19.3 linux/arm64)

Status:         4 Online, 0 Offline. 
API: https://minio.default.svc.cluster.local 
Console: https://10.244.2.13:9443 https://127.0.0.1:9443   

Documentation: https://min.io/docs/minio/linux/index.html
  1. Then access Console once again and check your file is still there:

Removed the services of the tenant, and got recreated automatically:

$ kubectl port-forward svc/cesar-console         -n default       9443:9443

Image is still there after the upgrade:

Screenshot 2022-11-11 at 3.44.50 PM.png