Synchronizing and securing your work fileset
Thursday, July 8th, 2010If you are in software development a little bit longer or you’re just a geek, chances are high that you’re doing your development on multiple machines and also on the road. There are two (well known) problems that come with this:
- Synchronization of the files on the different machines
- Securing your work so in case your laptop is stolen nothing is compromised (There are lot’s of high profile horror stories of stolen customer data)
The usual answer to problem 1. is a centra file server or NAS (Network Attached Storage) which you deploy in your intranet. If you want to work on the road you most likely have to throw some VPN or SSH-Tunnel at the problem.Problem 2 is usually solved by encryption of the hard drive or at least one partition.
My issues with solution 1 were twofold: I wasn’t always-on and even if I was: Access through my wireless plan was pretty slow especially for many small files. I ended up copying files on my hard disk for faster access but this defeated the purpose of the central storage completely since I sometimes forgot to copy the file back. Atfer some back and forth I came up with the following two tools:
Unison allows synchronization of a set of files in two directions. I basically have a central server at home where all the clients synchronize with. You can think of Unison as a VCS (Version Control System) without the versioning. It does not store old copies of the resources. The synchronization is done using the Rsync algorithm which basically works like that:
- The two parties exchange a list of files. This list can include timestamps, filesizes or even hash sums.
- If two files differ and the filesize is over a certain thereshold both parties chunk the file and submit a list of hash sums of the chunks using a rolling checksum (You can find the details on the Rsync website)
- The missing chunks are then transferred over the wire
This allows me to sync even changes in large files quite quickly. Thanks to SSH the transfer is secure and doesn’t need special VPN software.
TrueCrypt allows me to create an encrypted partition. This partition holds all my work data and this is the data I regularly sync with my central server.
This way I always have all the important stuff with me without compromising security or access speed. So if you ever consider setting up a VPN I would strongly suggest to give my solution a try instead. It’s much easier to maintain and doesn’t suffer from the always-on syndrome.
