Automating OS X app test build distribution across multiple OS versions

by Fraser Hess

With Apple now shipping OS X upgrades every 12-15 months, Mac developers are very quickly finding themselves supporting their apps on multiple OS X versions. Until recently, my approach to testing on multiple OS X versions involved partitioning an external USB drive and installing the OS X versions onto it and booting off the partitions to test.

While this approach is inexpensive, the test-discover-reboot-fix-build-test cycle just got to be too much for even this frugal Scot.

Enter the Mac mini

My new solution has 2 parts: a maxed out Mac mini with VMware Fusion to support all the targeted OS X versions and a scripted piece that will sync new test builds to the VMs.

The current $799 Mac mini has a quad-core Intel Core i7, a 1TB HD and is expandable to 16GB of RAM. Our local MicroCenter had a $749 sale on that model so I picked one up along with a 16GB RAM upgrade. After I setup 4 VMs, one for each of 10.6 – 10.9, I found the performance to be incredibly slow. Analyzing the issue quickly identified the speed of the stock HD as a bottleneck, so after considering returning the whole kit for a BTO Fusion Drive model, I picked up an OWC 240GB Mercury EXTREME™ Pro 6G SSD, and now the performance is great. (Installing an extra HD in a Mac mini is possible, but if you do, make sure you watch and understand the videos on the topic and make sure you order the correct parts to install in your Mac mini.)

Moving the builds around

Each of my Mac Xcode projects now has an extra scheme called MyApp test build. The following scheme settings are used: In Run MyApp.app, I’m using the Release build configuration and I’ve set Launch to Wait for MyApp.app to launch since I don’t intend to run these builds from Xcode. Most importantly, in Build->Post Actions, I’ve added a Run Script Action with Provide build settings from set to the app. The script below creates a new uniquely-named folder in the $root_destination_folder every time you build and will copy the resulting product into that newly created folder. The folder name combines date/time, product name, and the current git describe. The $root_destination_folder should be in a folder that is synced by a service. I used Dropbox at first, but it doesn’t seem optimized for the large number of small files that compose a Mac application bundle, even with LAN syncing turned on. I’m now using BitTorrent Sync, which uses the BitTorrent protocol. Even as a beta release, I had great success with it. By adding the shared folder to the BitTorrent Sync client on each test VM, every time I make a test build, it automatically appears on each VM.

By running multiple OS X versions in VMs simultaneously and syncing test builds, I’ve nearly eliminated all the waiting involved in testing and iterating across multiple OS X versions.

date=`date +%Y%m%d-%H%M%S`
version=`$git describe --dirty`
mkdir -p $full_dir_path

Note 1: For testing on Mac OS X 10.6 Snow Leopard, only Snow Leopard Server is supported in VMware Fusion. Apple has made Snow Leopard Server available to paid Mac developers as a free download. The provided serial number expires at the end of 2014.