minio vm‐broker replication - allanrogerr/public GitHub Wiki
Login
Create instances source and target
Enable TLS while creating them, or afterwards in Edit
or
On each instance, download certgen and create certificates
cd $HOME
wget https://github.com/minio/certgen/releases/latest/download/certgen-linux-amd64
chmod +x certgen-linux-amd64
./certgen-linux-amd64 -host "127.0.0.1,source.lab.min.dev,target.lab.min.dev"
mv public.crt $HOME/.minio/certs
mv private.key $HOME/.minio/certs
On each instance, download and run minio
cd $HOME
wget https://dl.min.io/server/minio/release/linux-amd64/minio
chmod +x minio
mkdir data
./minio server data --address :9000 --console-address :9090 &
On each instance, download and run mc
cd $HOME
mkdir mc
cd mc
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc alias set myminio-source https://source.lab.min.dev:9000 minioadmin minioadmin --insecure
./mc alias set myminio-target https://target.lab.min.dev:9000 minioadmin minioadmin --insecure
Review setup
./mc admin replicate add myminio-source myminio-target --insecure
./mc admin replicate info myminio-source --insecure
./mc admin replicate status myminio-source --insecure
Add buckets and content and observe replication
Source
Target
Try the same on clusters - create clusters source, jump and dest. Note the environment variable _MINIO_REVERSE_PROXY
_MINIO_REVERSE_PROXY=1 MINIO_CI_CD=1 MINIO_BROWSER_REDIRECT_URL="https://source0.lab.min.dev:9090" MINIO_SERVER_URL="https://source0.lab.min.dev:9000" \
./minio server \
https://source{0...7}.lab.min.dev:9000/tmp/data{1...2} \
--address :9000 --console-address :9090 --certs-dir $HOME/.minio/certs
_MINIO_REVERSE_PROXY=1 MINIO_CI_CD=1 MINIO_BROWSER_REDIRECT_URL="https://jump0.lab.min.dev:9090" MINIO_SERVER_URL="https://jump0.lab.min.dev:9000" \
./minio server \
https://jump{0...7}.lab.min.dev:9000/tmp/data{1...2} \
--address :9000 --console-address :9090 --certs-dir $HOME/.minio/certs
_MINIO_REVERSE_PROXY=1 MINIO_CI_CD=1 MINIO_BROWSER_REDIRECT_URL="https://dest0.lab.min.dev:9090" MINIO_SERVER_URL="https://dest0.lab.min.dev:9000" \
./minio server \
https://dest{0...7}.lab.min.dev:9000/tmp/data{1...2} \
--address :9000 --console-address :9090 --certs-dir $HOME/.minio/certs
On a separate instance, download and run mc
cd $HOME
mkdir mc
cd mc
wget https://dl.min.io/client/mc/release/linux-amd64/mc
chmod +x mc
./mc alias set myminio-source https://source0.lab.min.dev:9000 minioadmin minioadmin
./mc alias set myminio-jump https://jump0.lab.min.dev:9000 minioadmin minioadmin
./mc alias set myminio-dest https://dest0.lab.min.dev:9000 minioadmin minioadmin
See https://min.io/docs/minio/linux/reference/minio-mc-admin/mc-admin-replicate.html
Create or expand a site replication configuration. The configuration uses asynchronous site replication by default, as MinIO recommends.
./mc admin replicate add myminio-source myminio-jump myminio-dest #Output
Requested sites were configured for replication successfully.
Returns information about the sites in the site replication configuration.
./mc admin replicate info myminio-source
SiteReplication enabled for:
Deployment ID | Site Name | Endpoint | Sync | Bandwidth
| | | | Per Bucket
a9fd80db-8b35-47fc-a96f-4383172ed6fa | myminio-dest | https://dest0.lab.min.dev:9000 | | N/A
b06782f7-7daf-4e6e-aa4b-5bacfbb48f7d | myminio-jump | https://jump0.lab.min.dev:9000 | | N/A
1fd12738-4f91-43b3-aa7b-8bbaa6f7a10e | myminio-source | https://source0.lab.min.dev:9000 | | N/A
These give extra info
./mc admin replicate info myminio-jump
./mc admin replicate info myminio-dest
Update configuration to enable synchronous replication
./mc admin replicate update myminio-source --mode sync --deployment-id a9fd80db-8b35-47fc-a96f-4383172ed6fa --endpoint https://dest0.lab.min.dev:9000
./mc admin replicate update myminio-source --mode sync --deployment-id b06782f7-7daf-4e6e-aa4b-5bacfbb48f7d --endpoint https://jump0.lab.min.dev:9000
These are not needed
./mc admin replicate update myminio-jump --mode sync
./mc admin replicate update myminio-dest --mode sync
#Displays the status of the sites, buckets, users, groups, or policies for a site replication configuration.
./mc admin replicate status myminio-source
./mc admin replicate status myminio-jump
./mc admin replicate status myminio-dest
#Upload data
./mc cp --recursive /Users/allanreid/Documents/MinIO/demo/ myminio-source/data
#Rerun status check ./mc admin replicate status myminio-source
Bucket replication status:
● 1/1 Buckets in sync
Policy replication status:
● 5/5 Policies in sync
User replication status:
No Users present
Group replication status:
No Groups present
Object replication status:
Replication status since 19 minutes
dest0.lab.min.dev:9000
Replicated: 23 objects (152 MiB)
Transfer Rate: 0 B/s (avg: 904 kB/s; max 5.5 MB/s)
Latency: 17ms (avg: 11ms; max 108ms)
Link: ● online (total downtime: 0 seconds)
Errors: 0 in last 1 minute; 0 in last 1hr; 0 since uptime
jump0.lab.min.dev:9000
Replicated: 23 objects (152 MiB)
Transfer Rate: 0 B/s (avg: 960 kB/s; max 5.5 MB/s)
Latency: 17ms (avg: 10ms; max 111ms)
Link: ● online (total downtime: 0 seconds)
Errors: 0 in last 1 minute; 0 in last 1hr; 0 since uptime
Summary:
Replicated: 46 objects (304 MiB)
Queued: ● 0 objects, (0 B) (avg: 0 objects, 0 B; max: 0 objects, 0 B)
Received: 0 objects (0 B)
./mc admin replicate status myminio-jump
./mc admin replicate status myminio-dest
Extra
Resynchronizes data from one site in the replication configuration to a second site in the replication configuration in the event of lost data.
./mc admin replicate resync start myminio-source myminio-jump
./mc admin replicate resync status myminio-source myminio-jump
./mc admin replicate resync cancel myminio-source myminio-jump
Increase scanner speed. Check replication scanner status
./mc admin config set myminio-source api replication_priority=fast
./mc admin scanner status myminio-alpha/ -n 5