minio vm‐broker minio lab considerations - allanrogerr/public GitHub Wiki

Considerations for running labs on datacenter

  1. Consider only the usable higher port numbers from https://github.com/allanrogerr/minio-lab-terraform/blob/main/ansible/roles/haproxy/templates/haproxy-datacenter.cfg.j2

  2. Only the following port ranges are usable:

  • https: 443, 9000..9099, 30000..30099
  • http: 80, 10000..10099, 31000..31099
  • ssh,vnc: 20000..20099

Note: As seen above, nodeports for k8s on http/s can be taken from the range 30000..30099, 31000..31099

  1. linger needs to be enabled to persist between reboots or session logouts. This allows users who are not logged in to run long-running services such as with kubernetes. e.g.
loginctl enable-linger ubuntu
  1. Issue with podman
➜ docker run -itd \                                                                                               
   -p 10050:9000 \
   -p 10051:9090 \
   --name minio1 \
   -v /mnt/data/minio1:/data \
   -e "MINIO_ROOT_USER=minioadmin" \
   -e "MINIO_ROOT_PASSWORD=minioadmin" \
   -e "MINIO_PROMETHEUS_URL=http://prom.training0.minio.training:10070" \
   -e "MINIO_PROMETHEUS_JOB_ID=minio-job" \
   quay.io/minio/minio server /data --console-address ":9090"
Error: creating container storage: creating read-write layer with ID "f018abe30704d95279f60d7ddf52bfeba241506974b498717b872c0fa1d1df41": Stat /home/ubuntu/.local/share/containers/storage/vfs/dir/a8dfe82502f73a5b521f8b09573c6cabfaedfd5f97384124150c5afd2eac1d3a: no such file or directory

Solution: Run podman system prune --all --force && podman rmi --all

Issues

Issue: podman-compose does not respect the networks directive This means that for example in the IAM lab, the ldap server in its docker container will not be able to communicate with the minio server process running on its respective container.

More info:

Solution: use the host LXC container ip address to communicate between the ldap server and the minio server. To do this:

  1. before creating the ldap docker container, change the components."org.keycloak.storage.UserStorageProvider".config.connectionUrl ip address to the LXC host private IP address obtained by the following command: echo $(ip -4 addr show eth0 | grep -oP '(?<=inet\s)\d+(\.\d+){3}')

  2. use the same LXC host private IP address in the server_addr parameter of the mc idp ldap add command