Recently I lost a bunch of data from a MacBook Pro that spun off of the corner of my sofa onto my hardwood floor. Ouch. My last backup had been about a month earlier so I lost some pretty important stuff I was working on at the time and had to spend hours mungeing the drive with various data recovery tools.
I decided once and for all to solve my backup problems. Any backup strategy that I would employ, fundamentally, would have to adapt to the fact that I am lazy and that I am a data packrat. Here’s what I did, and this will work for anyone with OS X or linux, half an hour, and a unix host somewhere that has RSYNC installed properly.
First, I created a dedicated backup account on my DreamHost box (“username”) and gave it shell/ssh access. DreamHost is one of the few services that gives you SSH and Web hosting on a non-dedicated box in my limited searching, but more importantly their basic package gives you 200GB of free storage, which blows away pricing from any of the dedicated OS X backup companies. To create the key and place it on the server I did the following:
From the Command Line (on my machine):
ssh-keygen -d
# hit enter three times and replace “hostname” with your own
ssh username@hostname.dreamhost.com ‘test -d .ssh || mkdir -m 0700 .ssh ; cat >> .ssh/authorized_keys && chmod 0600 .ssh/*’ < ~/.ssh/id_dsa.pub
# enter your login password
That sets up key-based authentication so that when sshing to the host in the future, it won't need my password.
I then created a script on my local machine to back up the folders using rsync, one line per folder to backup. I find rsync extraordinarily complex to do actual syncing but when it comes to just pumping data out the back door on a regular basis, it's super-duper easy. Back to the command line and enter the following:
mkdir ~/Scripts
# makes a Scripts folder in your home directory
cat > ~/Scripts/sync-all
rsync -a ~/Documents username@hostname.dreamhost.com:~/Backup/
rsync -a ~/Library username@hostname.dreamhost.com:~/Backup/
rsync -a ~/Projects username@hostname.dreamhost.com:~/Backup/
rsync -a ~/Movies username@hostname.dreamhost.com:~/Backup/
rsync -a ~/Music username@hostname.dreamhost.com:~/Backup/
^D # (that’s a control-D) … now you’ve created a script file that backs up those folders
chmod a+x ~/Scripts/sync-all
# fixes the permissions on the file
… later I wanted to get fancier and wrote a script that also recorded the time and logged each backup … just replace the content of your simpler script file (above) with this using any text editor:
date > ~/Scripts/sync-all.log
rsync -av ~/Documents username@hostname.dreamhost.com:~/Backup/ >> ~/Scripts/sync-all.log
rsync -av ~/Library username@hostname.dreamhost.com:~/Backup/ >> ~/Scripts/sync-all.log
rsync -av ~/Projects username@hostname.dreamhost.com:~/Backup/ >> ~/Scripts/sync-all.log
rsync -av ~/Pictures username@hostname.dreamhost.com:~/Backup/ >> ~/Scripts/sync-all.log
rsync -av /Applications username@hostname.dreamhost.com:~/Backup/ >> ~/Scripts/sync-all.log
rsync -av ~/Incoming username@hostname.dreamhost.com:~/Backup/ >> ~/Scripts/sync-all.log
rsync -av ~/Scripts username@hostname.dreamhost.com:~/Backup/ >> ~/Scripts/sync-all.log
date >> ~/Scripts/sync-all.log
I then downloaded and installed LingOn, which lets you schedule and manage the rsync process via a GUI:
Here are some screenshots of the config process (this will rsync my Documents folder up to my Dreamhost acct every 10hrs, and do it as low priority so it doesn’t impact other things) but you can season to taste.
Enjoy! Hope this is useful. Thanks to Gersham for kicking this off.
-Ian.
Oh, I forgot to add one thing: For many different hosting services (such as Dreamhost) you need to set restrictive permissions on the directory you’ll be backing up files to:
chmod 700 ~/Backup
.. works fine. ALSO: if you want to use the -E command to backup Applications, then the structure would be something like:
rsync -aE ~/Projects username@hostname.dreamhost.com:~/Backup/
-Ian.
Please note that this approach will lose mac resource forks and other file metadata. For example, a *.webloc file will be corrupted.
Apple *partially* fixed the problem and patched rsync in 10.4 to add the -E option, but the rsync server must be running the same patch.
This will show you which files have resource forks:
find . -type f -exec test -s {}/..namedfork/rsrc \; -print
rsync’s awesome! Mozy’s pretty good too if you are willing to pay for backups or simply have a need for an enterprise level solution.
p.s. Ian can you reply to my email?
In a strange coincidence, Om wrote about a bunch of more tightly-integrated solutions today.
I am not a big fan of using WebDAV and manually mounting the remote host as a local drive, however. Doesn’t work in practise.
This method is ultra-cheap and 100% reliable, as long as you have your computer on when it’s time for the script to run.