MinIO and Koperator - cniackz/public GitHub Wiki

Objective:

Notify an image got uploaded via Kafka in k8s environment with Koperator and MinIO.

Previous Steps:

Clone Operator Repository:

git clone https://github.com/minio/operator.git
cd operator

Steps:

Create the cluster

kind delete clusters kind
kind create cluster --config ~/operator/testing/kind-config.yaml

Install cert manager

kubectl create -f https://github.com/jetstack/cert-manager/releases/download/v1.6.2/cert-manager.yaml

Install Zookeeper:

helm repo add pravega https://charts.pravega.io
helm repo update
helm install zookeeper-operator --namespace=zookeeper --create-namespace pravega/zookeeper-operator

kubectl create --namespace zookeeper -f - <<EOF
apiVersion: zookeeper.pravega.io/v1beta1
kind: ZookeeperCluster
metadata:
    name: zookeeper
    namespace: zookeeper
spec:
    replicas: 1
EOF

Install Koperator:

kubectl create --validate=false -f https://github.com/banzaicloud/koperator/releases/download/v0.21.2/kafka-operator.crds.yaml
helm repo add banzaicloud-stable https://kubernetes-charts.banzaicloud.com/
helm repo update
helm install kafka-operator --namespace=kafka --create-namespace banzaicloud-stable/kafka-operator

Create Kafka Cluster:

kubectl create -n kafka -f https://raw.githubusercontent.com/banzaicloud/koperator/master/config/samples/simplekafkacluster.yaml

Create the topic, wait until all above is ready:

cat << EOF | kubectl apply -n kafka -f -
apiVersion: kafka.banzaicloud.io/v1alpha1
kind: KafkaTopic
metadata:
    name: my-topic
spec:
    clusterRef:
        name: kafka
    name: my-topic
    partitions: 1
    replicationFactor: 1
    config:
        "retention.ms": "604800000"
        "cleanup.policy": "delete"
EOF

You should see:

$ cat << EOF | kubectl apply -n kafka -f -
> apiVersion: kafka.banzaicloud.io/v1alpha1
> kind: KafkaTopic
> metadata:
>     name: my-topic
> spec:
>     clusterRef:
>         name: kafka
>     name: my-topic
>     partitions: 1
>     replicationFactor: 1
>     config:
>         "retention.ms": "604800000"
>         "cleanup.policy": "delete"
> EOF
kafkatopic.kafka.banzaicloud.io/my-topic created

Install Ubuntu Pod:

cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Pod
metadata:
  name: ubuntu
  labels:
    app: ubuntu
spec:
  containers:
  - image: ubuntu
    command:
      - "sleep"
      - "604800"
    imagePullPolicy: IfNotPresent
    name: ubuntu
  restartPolicy: Always
EOF

Prepare Ubuntu Pod:

Make sure to get the IP Address from Kakfa pod prior next commands, example: 10.244.2.5

apt-get update
apt-get install kafkacat
apt-get install wget
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
mv mc /usr/local/bin/mc
kafkacat -C -b 10.244.2.5:29092 -t my-topic

Install the MinIO Operator:

kubectl apply -k ~/operator/resources

Install MinIO Tenant:

kubectl apply -k ~/operator/examples/kustomization/tenant-lite

Using an Ubuntu Pod inside the cluster, configure Kafka notification, restart server and upload the image:

Make sure to get Kafka IP Address for command below, example: 10.244.2.5

mc alias set myminio https://minio.tenant-lite.svc.cluster.local minio minio123

mc admin config set myminio notify_kafka:1 tls_skip_verify="off"  queue_dir="" queue_limit="0" sasl="off" sasl_password="" sasl_username="" tls_client_auth="0" tls="off" client_tls_cert="" client_tls_key="" brokers="10.244.2.5:29093" topic="my-topic" version="" --insecure

mc admin service restart myminio --insecure

touch rose.jpg
echo "a" > rose.jpg

mc mb myminio/images --insecure
mc event add  myminio/images arn:minio:sqs::1:kafka --suffix .jpg
mc event list myminio/images

mc cp rose.jpg myminio/images --insecure

Look at consumer, you should see the notification:

{"EventName":"s3:ObjectCreated:Put","Key":"images/rose.jpg","Records":[{"eventVersion":"2.0","eventSource":"minio:s3","awsRegion":"","eventTime":"2022-08-08T18:57:45.129Z","eventName":"s3:ObjectCreated:Put","userIdentity":{"principalId":"minio"},"requestParameters":{"principalId":"minio","region":"","sourceIPAddress":"10.244.2.19"},"responseElements":{"content-length":"0","x-amz-request-id":"1709739F2A493AB1","x-minio-deployment-id":"bba6bc73-d95c-4039-bb20-271ca323aec1","x-minio-origin-endpoint":"https://minio.tenant-lite.svc.cluster.local"},"s3":{"s3SchemaVersion":"1.0","configurationId":"Config","bucket":{"name":"images","ownerIdentity":{"principalId":"minio"},"arn":"arn:aws:s3:::images"},"object":{"key":"rose.jpg","size":2,"eTag":"60b725f10c9c85c70d97880dfe8191b3","contentType":"image/jpeg","userMetadata":{"content-type":"image/jpeg"},"sequencer":"1709739F2B7C7BC2"}},"source":{"host":"10.244.2.19","port":"","userAgent":"MinIO (linux; amd64) minio-go/v7.0.31 mc/RELEASE.2022-08-05T08-01-28Z"}}]}

MinIO and Koperator - cniackz/public GitHub Wiki

Related links:

Objective:

Previous Steps:

Steps: