OSX Virtual Machine Setup

Introduction

Our goal with creating virtualized OSX build and test systems is to allow us to backup, restore and archive the build OSX state for CASA releases. This allows us to retireve a particular OSX operating system image and create patches for previous CASA releases. This is not typically done for old releases, but it is not unusual for patches to be created for the last CASA release. By the time a patch is created, third-party packages which support building CASA for the upcoming release. Virtualized OS images are a nice solution for this problem.

Our reading of the Apple OSX EULA is that it allows up to three versions of OSX to be installed on Apple hardware in expectation that this procedure will be useful for development and testing of new software which runs on OSX. Our approach is to install Linux on the base Apple hardware and run three virtualized OSX images on QEMU/KVM. Our hardware currently consists of several Mac Minis and one Mac Pro. The Mac Pro can easily run Linux and three OSX guests. A Mac Mini is capable of running Linux plus one virtual guest.

Boot Loader

Yosemite (OSX 10.10), El Capitan (OSX 10.11) and Sierra (OSX 10.12) all require an intermediate boot loader. This boot loader acts as a bridge between SeaBIOS of QEMU/KVM and the BIOS expected by Apple (EFI BIOS). The first task when considering adding a new OSX release is to check to see if people have had success using the bootloader used with the previous version of OSX or if a new boot loader is required.

The older bridge BIOS (chameleon) is usable by releases up to and including 10.10 but with 10.10 this development was forked and a new boot loader was created (enoch) for El Capitan. This newer boot loader was modified to support Sierra. We obtain the boot loader by downloading the Chameleon (note v) package and extracting the boot loader from it (note i). The application we used to extract the boot loader from the Chameleon package is Pacifist (note vii). After extracting the boot loader, we give it a descriptive name and save it for later use by QEMU.

Bootable Image

Apple stopped distributing OS DVDs with Snow Leopard (OSX 10.6). Since then, they only distribute installation apps (see note vi). One of the most important aspects to running OSX on QEMU/KVM is creating a good ISO image from which to install. The reasons we use ISO images instead of some other image are:

  • ISO is read-only
  • ISO provides universal readability
  • ISO supported by all hardware (virtual or physical)

Unfortunately, creating a good ISO boot image for OSX is harder than one would like, and the layout of each of the OSX Installer apps makes this error prone.

Some symlinks must be replaced with actual files and the kernel image must be extracted from a pkg and inserted into the right place. For this, Pacifist (note vii) is used to extract the "Kernels" folder, and "create_osx_install_iso.sh" (note viii), run as root on a physical OSX system, is used to create the ISO:

  1. extract the "Kernels" folder from the OSX install app you want to create an ISO for into the current directory (use Pacifist search to find the folder)

  2. ./create_osx_install_iso.sh --app /Applications/Install\ OS\ X... --iso osx.iso --method 2 --verify (see notes i, ii)

Install Disk

Once you have a good ISO image, the next step is to create a "qcow2" disk image to be used for the OSX install. This can be done from the KVM virtual manager GUI or from the command line:

qemu-img create -f qcow2 /home/kvm-images/yose.qcow2 80G

The ISO's OS can be installed on the newly created virtual disks with the commands below (labeled Yosemite and El Capitan) with the appropriate bridge BIOS (notes ix,x). Once you get past accepting the Apple license agreement, use the Utilities menu item to open "Disk Utility" to format the virtual disk for the install (see notes iii, iv). The 64bit key ($OSK) can be rerieved using the SmcDumpKey program (note iii).

  • Yosemite

    1. /bin/qemu-system-x86_64 -enable-kvm -m 8192 -cpu core2duo,vendor=GenuineIntel -machine q35 -usb -device usb-kbd -device usb-mouse -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -kernel /root/kvm-osx/chameleon_svn2783_boot -smbios type=2 -device ide-drive,bus=ide.2,drive=MacHDD -drive id=MacHDD,if=none,file=/home/kvm-images/yose.qcow2 -netdev user,id=hub0port0 -device e1000-82545em,netdev=hub0port0,id=vnet0 -monitor stdio -smp 4,cores=2 -device ide-drive,bus=ide.0,drive=MacDVD -drive id=MacDVD,if=none,snapshot=on,file=/root/kvm-osx/Yosemite.iso

  • El Capitan

    1. /bin/qemu-system-x86_64 -enable-kvm -m 8192 -cpu core2duo,kvm=off,vendor=GenuineIntel -machine q35 -usb -device usb-kbd -device usb-mouse -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -kernel /root/kvm-osx/enoch_svn2795_boot -smbios type=2 -device ide-drive,bus=ide.2,drive=MacHDD -drive id=MacHDD,if=none,file=/home/kvm-images/elcap.qcow2 -netdev user,id=hub0port0 -device e1000-82545em,netdev=hub0port0,id=vnet0 -monitor stdio -smp 4,cores=2 -device ide-drive,bus=ide.0,drive=MacDVD -drive id=MacDVD,if=none,snapshot=on,file=/root/kvm-osx/ElCapitan.iso

    2. at "boot:" prompt in QEMU window, enter:

      "KernelBooter_kexts"="Yes" "CsrActiveConfig"="103"

  • Sierra

    1. /bin/qemu-system-x86_64 -enable-kvm -m 4096 -cpu Penryn,kvm=off,vendor=GenuineIntel -machine q35 -usb -device usb-kbd -device usb-mouse -device isa-applesmc,osk="ourhardworkbythesewordsguardedpleasedontsteal(c)AppleComputerInc" -kernel /opt/kvm-disks/enoch_svn2839_boot -smbios type=2 -device ide-drive,bus=ide.2,drive=MacHDD -drive id=MacHDD,if=none,file=/opt/kvm-disks/SierraRel.qcow2 -netdev tap,id=vnet0 -device e1000-82545em,netdev=vnet0,mac=52:54:00:C6:C6:15 -monitor stdio -smp 6,cores=3, -device ide-drive,bus=ide.0,drive=MacDVD -drive id=MacDVD,if=none,snapshot=on,file=/opt/iso-images/Sierra-rel.iso

    2. at "boot:" prompt in QEMU window, enter:

      "KernelBooter_kexts"="Yes" "CsrActiveConfig"="103"

Risks

We have found this approach to be very beneficial. The inability to encapsulate and archive build state previously caused development delays. In general, the KVM/QEMU approach works very well, and we are currently satisfied that this is a good approach. However, there are some inherent risks involved:

  • we depend upon the internet community to develop bootloader code
  • Apple often introduces incompatibilities with new OSX releases
  • Apple could change there EULA at some point

Notes

  1. https://macosxvirtualmachinekvm.wordpress.com/guide-mac-os-x-10-11-el-capitan-vm-on-unraid/

  2. http://www.tonymacx86.com/threads/install-os-x-el-capitan-with-chameleon.179905/

  3. http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/

  4. https://blog.ostanin.org/2014/02/11/playing-with-mac-os-x-on-kvm/

  5. http://www.insanelymac.com/forum/files/file/71-enoch/

  6. It is important to download the install app each time a new verson of OSX is released because this preserves your right to download it in the future, even after a newer version of OSX is released. If you have not downloaded version A of OSX by the time verson B is released, you are out of luck WRT version A.

  7. http://www.charlessoft.com/

  8. https://github.com/kholia/OSX-KVM

  9. "chameleon_svn2783_boot" is found at http://www.contrib.andrew.cmu.edu/~somlo/OSXKVM/

  10. "Core.pkg/usr/standalone/i386/boot" is extracted from http://www.insanelymac.com/forum/files/file/71-enoch/ to create "enoch_svn2795_boot" as described in note i

  11. http://www.insanelymac.com/forum/topic/309087-insanely-fast-virtual-mac-qemu-ovmf-clover-and-native-graphics/page-13