Use rsync to mirror websites
tl;dr
apt update && apt install rsync
#Access via remote shell:
# Pull: rsync [OPTION...] [USER@]HOST:SRC... [DEST]
# Push: rsync [OPTION...] SRC... [USER@]HOST:DEST
# Pull: sync remote with local
rsync -vhaze ssh user@server.example.com:/var/www/ /var/www
rsync -vPhaze "ssh -p 1234 -i /root/.ssh/id_rsa" root@server.example.com:/var/www/cakebox.me/public_html/ /var/www/cakebox.me/public_html
rsync -qaze "ssh -p 1234 -i /root/.ssh/id_rsa" root@server.example.com:/etc/letsencrypt/archive :/etc/letsencrypt/live :/etc/letsencrypt/renewal /etc/letsencrypt/
-v, --verbose increase verbosity
-h, --human-readable output numbers in a human-readable format
-q, --quiet suppress non-error messages
-a, --archive archive mode; equals -rlptgoD (no -H,-A,-X)
-z, --compress compress file data during the transfer
-e, --rsh=COMMAND specify the remote shell to use
--progress show progress during transfer
-P same as --partial --progress
- explainshell for the rsync command.
-e
is basically the shell to use, e.g.ssh -p 2234 -i /user/.ssh/id_rsa
, followed by the command-a
is used for preservation (ownership, permissions, soft links etc.)- You can specify custom SSH port and SSH key to use for the connection. The path for the key file needs to be absolute,
~
will not expand, so you have to use/userhome
- Make sure which way you’re syncing. If note sure, you could end up overwriting imprtant files. For example config files with database connection details for example, this has happened.
- An easy way to determine which way you are syncing is the
-e
flag.-e
determines the remote shell, so if you’re using the option, you’re pulling, i.e. syncing remote changes to local. -e
can not be used when you’re syncing local changes to remote
Create an unprivileged user for the sake of transferring files
useradd -d /home/rsyncuser -m -s /bin/bash rsyncuser
passwd rsyncuser
Create and copy an SSH key for password-less access
# Generate, copy and connect with an SSH key
ssh-keygen -t rsa
ssh-copy-id -i /home/
rsyncuser/.ssh/id_rsa.pub rsyncuser@webserver.example.com
ssh rsyncuser@webserver.example.com
Create a cronjob to automate the whole thing
crontab –e
*/5 * * * * rsync -vhaze "ssh -p 1234 -i /root/.ssh/id_rsa" root@server.example.com:/var/www/cakebox.me/public_html/ /var/www/cakebox.me/public_html