Small Office Backup with Rsync | January 23, 2007
When setting up our new office, I wanted to ensure everything was backed up correctly. I asked around for backup solutions, but the options were overwhelming. As we were a new company, I didn’t want to spend a huge amount of money on complicated software or hardware solutions, so in the end we went with something that’s already built into the operating system–Rsync.
This small but powerful command line tool forms the basis for a lot of Mac backup solutions, which are essentially GUI front ends. Rsync is much loved by techies, but I’m no Unix wizard soit took a while to get things set up. This quick tutorial outlines how I’ve got backups working at Clearleft. This is by no means a definitive guide, and I’m sure there are much better ways of doing it. So if you’ve got any better ideas, please let me know.
The first step was to find something to back-up to. We thought about network attached storage (NAS), but in the end went for the simple option of a Mac Mini connected to a removable hard drive. We have two such drives and rotate them weekly to ensure we have an offsite back-up.
What we’re going to do is set the Mac Mini up so it connects to each machine on the network at a set time of day, and then run an Rsync back-up. To connect to each machine you first need to give them a distinct IP address on your network.
Go into your network preferences, select the TCP/IP menu option and in the “Configure IPv4” dropdown, select “Using DCHP with manual address”. I’m not sure what the best IP numbering convention is, but we have all our desktops starting from 192.168.1.10, so other devices like routers, printers or laptops can grab the first 10 slots automatically if they want.
Once each machine has an IP address, you need to make sure the Mac Mini can connect to it over SSH. To do this, go into the sharing preferences and check the “Remote Login” option.
Now, lets create the backup command on the Mac Mini. I’ve created a new folder on the Mini called back-up where I’m keeping all my configuration files. Create a new text file in this folder and call it the a sensible name like andybak.command.
First you need to set all the required flags for the rsync command. I’m not going to go into them all, but if you’re interested you can type man rsync for the full list.
rsync -a -v -r -S -x -z --delete -e
The next thing you need to do is connect to the machine and folder you wish to backup using ssh
Now specify the target location of your backup. In our case it’s a mounted volume called “LaCie Disk”
Lastly we don’t want to back up everything, so I’m going to create an exclusions text file. Add a pointer to this text file next.
Save this file and create a new file for your excludes called andy_excludes.txt. In this file list all the folders you wish to exclude. I’ve got a lot of music on my machine so I’m going to exclude the music folder. If you have lots of movies or pictures, you may want to exclude those folders as well.
Save the textfile.
Now we can run the command and see if it works. If you want to be extra cautious there is a flag you can add to your command file that will run a simulation instead of the real thing. As this will be the first time you’ve run this command, the initial backup may take a while. To run the command, simply double click the file and it should launch and run in the terminal window.
The first thing this command will do is try to connect to the computer you’re backing up using SSH. Because this is the first time you’ve connected, it will ask you if you’re sure of the authenticity of the host. Type “yes” to proceed. You’ll next be asked the password of the host machine. Type it in now and the backup will start running. Go make a cup of tea as it may take a few minutes.
Once the back-up is complete, check that a new folder has been added to the backup drive and that all the selected files have been backed up.
Now you obviously don’t want to enter the password each time you run a backup, so you need to set up a public and private key on the backup machine, and then copy the public key over to the host machine. This is where things get a little tricky as there are numerous ways of doing this, some more secure than others. Luckily I did this ages ago, so I’m not even going to attempt to explain how this is done. If you’re interested, do a search on ssh or public key authentication on OS X.
On the Mac Mini, locate your public key. In our case the file was called id_rsa.pub and it was in a folder called .ssh. Using secure copy (scp), copy this key to the authorized_keys file in the .ssh folder on the machine you’re wanting to connect to. OIf the file or folder doesn’t exist, you will need to create it.
scp /Users/clearleft/.ssh/id_rsa.pub firstname.lastname@example.org:.ssh/authorized_keys
You’ll be asked for the password of the machine you’re connecting to. Once you’ve entered it, the files will copy over, and you’ll never be asked for a password again. To check the public key is working, run the backup command again and it should run without asking for a password.
We’re almost there. Just one last step in order to make the backups really useful. We need to automate their execution. To do this, you need to decide a time for each backup to run. We run ours in the evening when everybody is out of the office, to avoid the inevitable network slowdown. First, go into the energy saver preferences for the machine you’re backing up, click the “schedule” button and wake the machine up 5 minutes before you plan to run the backup.
Then go back to the Mac Mini and edit your crontab file.
sudo pico /private/etc/crontab
Set the time you want the command to run in minutes and hours, and leave the day, month ect starred out, so your backup runs every day. Under the command heading, add the path to your command along with an optional path to a log file.
/Users/clearleft/backup/andybak.command >> /Users/clearleft/backup/backup.log
Do this for every machine on your network, and every night you’ll have trouble free, automated backups.
Posted at January 23, 2007 1:02 PM