How to synchronize files between 2 Linux servers automatically

Few days ago, I planned to install piwigo to keep up my images out of an external drive. I don’t want to lose them, so I configured 2 servers as primary and secondary, the second one as backup.

Configuration

In this walkthrough, I will have 2 servers:

  • the primary, which will host piwigo, 37.187.119.x (Ubuntu)
  • the secondary, which will only act as backup server, 5.196.89.y (Ubuntu)

RSync installation

RSync, accronym for Remote Synchronization, is a synchronization tool. It will get files from a system (local or distant) and copy them to another system (local or distant), incrementally.

I planned to install RSync only on my secondary server, 5.196.89.y, which will act as a backup server. To install it, retrieve the command associated to your system, and execute it on your server.

$ sudo apt-get install rsync     [On Debian, Ubuntu and Mint]
$ sudo yum install rsync         [On RHEL/CentOS/Fedora and Rocky Linux/AlmaLinux]
$ sudo emerge -a net-mis/rsync   [On Gentoo Linux]
$ sudo pacman -S rsync           [On Arch Linux]
$ sudo zypper install rsync      [On OpenSUSE]

If you try to execute “rsync -V”, it will show you the installed version.

Using RSync to Syncing directory files

Rsync is installed, it’s time to use it to perform backups. You will need one directory on your primary server to sync, and one directory on your secondary server to store synchronized datas. In this example, I will use ~/datas on my primary server (which already have data), and ~/backup on my secondary server (which is empty). We can now proceed for a test from the secondary server, to check if everything is working fine

$ rsync -avzhe ssh ubuntu@37.187.119.x:/home/ubuntu/datas/ /home/ubuntu/backup
$ ls /home/ubuntu/backup

Next to those 2 commands, /home/ubuntu/backup on your secondary server must contains every file that exist in /home/ubuntu/datas on your primary server. I now have to configure the rsync command to not ask for credentials and schedule it.

SSH configuration

Connected on my secondary server, I used the following command to create ssh keys.

$ ssh-keygen -t rsa -b 2048

I pressed Enter when prompted for a passphrase, I didn’t need it, so it can be empty. Now I have my ssh keys, I need to send them to my primary server.

$ ssh-copy-id -i /home/ubuntu/.ssh/id_rsa.pub ubuntu@37.187.119.x

If I execute again the rsync command, it will not ask for password anymore. I only have to schedule this command to finalize my backup system. Still on my secondary server, I will configure crontab.

$ crontab -e

Crontab is opened in a editor. I will ask the following line to plan rsync process after every 5 minutes.

*/5    *    *     *     *  rsync -avzhe ssh ubuntu@37.187.119.x:/home/ubuntu/datas/ /home/ubuntu/backup

You can check that cron is executed by executing the following command.

$ grep CRON /var/log/syslog

All executions will be displayed.

Syncing two servers create the perfect solution to prevent system failure. If my primary go down, I will still have a copy of all my data, as the backup server will be hosting a complete copy.

Please follow and like us:

Leave a Reply

Your email address will not be published. Required fields are marked *

This site uses Akismet to reduce spam. Learn how your comment data is processed.