rclone setup - jigneshpshah/greycube_helpmanual GitHub Wiki
#Setup rclone on client server
-
Login as frappe user
-
sudo apt-get install zip
-
curl https://rclone.org/install.sh | sudo bash
-
rclone config
- keep backup server username/pwd ready
- when in doubt use default options.
- advance option is not required.
Options value
name>greycube_backup_server
Storage>28 (sftp)
host>127.0.0.1
-It should be server IP address
user>greycube_backup_user
port>
password > {keyinhere}
key_use_agent> false
use_insecure_cipher> true
disable_hashcheck> false
will look like this
[greycube_backup_server]
type = sftp
host = XX.123.123.223
user = greycube_backup_user
pass = *** ENCRYPTED ***
key_file_pass = *** ENCRYPTED ***
key_use_agent = false
use_insecure_cipher = true
disable_hashcheck = false
Quit and from terminal
Check if rclone config smoothly by listing directory of remote server
- rclone ls greycube_backup_server:
#Create backup_script.sh in frappe-bench folder
- nano backup_script.sh
#!/bin/bash
#server - bench level variables
rclone_remote_name="greycube_backup_server"
frappe_bench_path="/home/frappe/frappe-bench"
app_version_file='/app_version.json'
cd $frappe_bench_path
#for each site copy below snippet
# individual site variables
local_site_name=site1.local
local_site_backup_path="/home/frappe/frappe-bench/sites/site1.local/private/backups"
# convention backup/domain or server or client name / site name without special charachters , use _
remote_backup_path="backup/artfactory/site1_local"
echo "Backup Starts - "$local_site_name
/usr/local/bin/bench --site $local_site_name execute frappe.utils.change_log.get_versions > $local_site_backup_path$app_version_file
/usr/local/bin/bench --site $local_site_name backup --with-files
# delete all but 10 newest files
cd $local_site_backup_path
ls -1tr | head -n -10 | xargs -d '\n' rm -f --
cd $frappe_bench_path
rclone sync $local_site_backup_path $rclone_remote_name:$remote_backup_path
echo "Backup Ends- "$local_site_name
Check manually is it working
- Go to bench folder
- sh backup_script.sh
Setup cronjob
- Login as frappe user
- Check server time
- date "+%H:%M:%S %d/%m/%y"
- say it is 04:31:24 31/05/19
- Check client time using google
- say it is 6:31
- i.e. server is 2 hours behind. Hence cronjob should run at 10 PM that would be midnight in client country
- use https://crontab.guru/ to generate cron time entry
- 0 22 * * *
- Go to server using frappe user and set cronjob
- crontab -e
0 22 * * * /bin/sh /home/frappe/frappe-bench/backup_script.sh >> /home/frappe/frappe-bench/logs/autobackup_log.log 2>&1
- sudo cron service restart
Setup rclone for s3 Object on client server
RClone for Contabo S3 Object setup
#!/bin/bash
# https://satcit.me/index.php/2022/03/15/contabo-object-storage-rclone
#server - bench level variables
rclone_remote_name="greycube_backup_server_contabo2_s3"
frappe_bench_path="/home/frappe/frappe-bench"
frappe_sites_path="/home/frappe/frappe-bench/sites"
app_version_file='/app_version.json'
ip4=$(/sbin/ip -o -4 addr list eth0 | awk '{print $4}' | cut -d/ -f1)
#for each site
cd $frappe_sites_path
for f in */ ;
do
local_site_name=${f%/}
if [ "${local_site_name}" != "assets" ]; then
echo "$local_site_name"
cd $frappe_bench_path
local_site_backup_path="/home/frappe/frappe-bench/sites/${local_site_name}/private/backups"
remote_backup_path="${ip4}-${local_site_name}"
echo "Backup Starts - "$local_site_name
/usr/local/bin/bench --site $local_site_name execute frappe.utils.change_log.get_versions > $local_site_backup_path$app_version_file
/usr/local/bin/bench --site $local_site_name backup --with-files
# delete all but 7 newest files [1 backup has 4 files i.e. private/public/site_config/DB and app_version]
cd $local_site_backup_path
ls -1tr | head -n -7 | xargs -d '\n' rm -f --
cd $frappe_bench_path
# create bucket
rclone mkdir $rclone_remote_name:$remote_backup_path
rclone sync $local_site_backup_path $rclone_remote_name:$remote_backup_path
echo "Backup Ends- "$local_site_name
fi
done