Create A Raspberry Pi Automatic Network Backup Server

I have a few Raspberry Pi units that I’ve picked up over the years. As of now I have a RPi 256MB, RPi 512MB and a RPi2 1024MB unit. I’m currently using the faster model as a RetroPie which I’ll discuss in a future article, but for the older models I have them doing server stuff. For example, I have my 256MB model acting as a network backup server that one of my computers automatically uploads to.

Here we’re going to look at what it takes to get an automated backup server rolling with a Raspberry Pi and how exactly it can be useful to you.

If you haven’t already, it may be useful to you to check out my previous tutorial for configuring a Raspberry Pi as a headless system. This guide will be using Raspbian as the Linux flavor, but we’re not going to go through the process of operating system installation and configuration in this guide. It will be strictly relating to topics on backups and server configurations.

To handle our backups we’re going to be using a nifty Unix and Linux tool called rsync.

Rsync defined by Wikipedia:

Rsync is a widely-used utility to keep copies of a file on two computer systems the same. It is commonly found on Unix-like systems and functions as both a file synchronization and file transfer program.

Because we’re using rsync, Windows users should stop reading this guide because this backup solution is going to be too painful a process for you to get set up with. Linux and Mac users should stick around.

Let’s look at common usage of rsync between a source and destination machine. The source will be my Mac, but the destination will be the Raspberry Pi that contains Raspbian Linux.

rsync -az source destination --progress --delete

The above line will copy a file or directory from the source to the destination. The --progress flag indicates that we want to know the progress of the transfer because some files may be quite large. The --delete flag indicates that should we rsync and a source file no longer exists when it did previously, the remote copy will be removed. Finally the -a means archive and the -z means compress.

Let’s make this example a little more specific. Check out the following line:

rsync -az ~/Desktop pi@10.0.1.61:~/ --progress --delete

In the above example, I’d be uploading all files found on my desktop to the home directory of pi on the remote Raspberry Pi. This includes creating the directory Desktop on the Raspberry Pi itself.

My favorite part of rsync is that it only copies what has changed vs a standard SCP that will transfer files regardless making your backup process long and terrible.

This is cool so far, but there are a few inconveniences here:

1. We have to enter the password for the pi user every time we call the command
2. This is a manual process of backup

Let’s start by correcting the password issue. It can be easily resolved by setting up a public and private key pair on your local and remote machine.

Again, assuming you’re using Mac or Linux, execute the following from your Terminal:

ssh-keygen -t rsa

Leave the password blank when asked.

You should have two key files. Yes we didn’t add a password to the private key. It may not be the best in terms of security, but you could be in worse shape. Figure out what meets your needs on private key security.

We need to upload the public key to the Raspberry Pi server. From your Terminal execute the following:

cat /path/to/public/key/id_rsa.pub | ssh pi@10.0.1.61 "mkdir -p ~/.ssh && cat >>  ~/.ssh/authorized_keys"

Remember to swap out my 10.0.1.61 IP with that of your actual Raspberry Pi.

The above command will create an .ssh directory in your Raspberry Pi home directory and append the contents of id_rsa.pub to the authorized_keys file.

Try to SSH again. If all went well you should connect to the pi user without any form of password.

Next we want to automate this backup process. To do this we’re going to use cron for Linux and launchd for Mac to run scripts on a schedule. If you’re unfamiliar with cron and launchd, you essentially pick a re-occurring time and date and a script and it will run every time it hits regardless if you’ve shut off your computer between runs.

Before we design the cron and launchd schedule, let’s come up with a backup script called backup.sh:

rsync -az ~/Desktop pi@10.0.1.61:~/ --progress --delete
rsync -az ~/Pictures pi@10.0.1.61:~/ --progress --delete
rsync -az ~/Documents pi@10.0.1.61:~/ --progress --delete

Yes, the above is doing three different rsync transactions. You can certainly rig something together to do this all in one, but I find there to be nothing wrong with this approach.

Now that our script is made, we need to define our cron schedule for a Linux host. From the Terminal, execute the following:

crontab -e

Scheduling can be a tricky beast, but this might make it easier:

Minute    Hour    Day of Month    Month    Day of Week    Script

So let’s say we want to run our backups every thirty minutes of every day, of every month, of every week. We might plugin something like this in for an entry:

0,30    *    *    *    *    /path/to/backup.sh

This says it will execute on the zero minute and the thirty minute mark.

Now let’s look at how to do this if you’re on a Mac as your source machine. We need to create a file ~/Library/LaunchAgents/com.nraboy.backup.plist with the following XML:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>Label</key>
        <string>com.nraboy.backup</string>
        <key>ProgramArguments</key>
        <array>
            <string>/path/to/backup.sh</string>
        </array>
        <key>StartInterval</key>
        <integer>60</integer>
    </dict>
</plist>

You can change com.nraboy.backup if you don’t want to name it as such. Just make sure the filename matches the name given in the plist file. This file says that our backup scripts will run every sixty seconds.

To make this plist file work, we need to load it and start it like so:

launchctl load ~/Library/LaunchAgents/com.nraboy.backup.plist
launchctl start com.nraboy.backup

You should be good to go now!

If you got this far and everything is working, you might notice that larger backups are killing your network bandwidth. This is because by default rsync will try to transfer at the fastest speed possible, making all other network activities pretty much unusable while it is happening. You can fix this by setting a limit. For example you can do something like:

rsync -az source destination --bwlimit=5000 --progress --delete

By setting --bwlimit=5000 we are limiting the transfer to 5MB per second. You probably want to fiddle with this setting until you find a good spot. You wouldn’t want backups to be so slow that the next cycle comes around for another backup run.

Conclusion

The usage of rsync is not particularly difficult as it is just a fancy copy command. The time consuming part came in the form of task scheduling with cron and launchd scripts for Mac and Linux. If you wanted to take this guide to the next level you could enhance your backup.sh script to do full snapshots on certain events. Like maybe do standard backups hourly, but full snapshots organized by timestamps once weekly.

The great thing about having an rsync server on a Raspberry Pi is that you can lock it in a drawer or somewhere else that is out of sight because it is so small. Your backups will continue to happen for as long as the Raspberry Pi is powered on and connected to the internet.

Nic Raboy

Nic Raboy is an advocate of modern web and mobile development technologies. He has experience in C#, JavaScript, Golang and a variety of frameworks such as Angular, NativeScript, and Unity. Nic writes about his development experiences related to making web and mobile development easier to understand.