MinIO

Step 1: Set Up MinIO Gateway for S3 (GCP)

On your MinIO server, configure it as a gateway for GCP:

export MINIO_ACCESS_KEY="YOUR_GCP_ACCESS_KEY"
export MINIO_SECRET_KEY="YOUR_GCP_SECRET_KEY"

minio server --address ":9000" gcs http://storage.googleapis.com

Now MinIO acts as an S3-compatible layer over GCP.

Step 2: Configure Replication

On MinIO, add a replication rule:

mc alias set minio http://minio.example.com:9000 YOUR_MINIO_ACCESS_KEY YOUR_MINIO_SECRET_KEY
mc admin bucket remote add minio/do-bucket http://storage.googleapis.com YOUR_GCP_ACCESS_KEY YOUR_GCP_SECRET_KEY --region=us-central1 --sync

Then enable replication:

mc replicate add minio/do-bucket --remote-bucket gcp-bucket --replication-role "mirror"

This will sync changes in real-time from DigitalOcean Spaces to GCP.

---

rclone

Step 1: Configure DigitalOcean and GCP in rclone

Follow the same rclone config setup as before:

Add DigitalOcean Spaces as do_spaces. Add Google Cloud Storage (S3) as gcp_s3.

Step 2: Run rclone in Real-Time Sync Mode

Use the rclone sync command in a loop:

while true; do
  rclone sync do_spaces:my-do-bucket gcp_s3:my-gcp-bucket --progress
  sleep 10
done

This checks for new changes every 10 seconds. You can adjust the sleep time as needed.

Step 3: Run as a Background Service

Create a systemd service to keep it running:

sudo nano /etc/systemd/system/rclone-mirror.service

Paste this:

[Unit]
Description=Rclone Sync DigitalOcean Spaces to GCP S3
After=network-online.target

[Service]
ExecStart=/usr/bin/rclone sync do_spaces:my-do-bucket gcp_s3:my-gcp-bucket --progress
Restart=always

[Install]
WantedBy=multi-user.target

Then enable it:

sudo systemctl daemon-reload
sudo systemctl enable rclone-mirror
sudo systemctl start rclone-mirror