Docker is one of the tools I feel strongly makes life better, though it has an incredibly steep learning curve. The process of building and using containers is difficult to explain, I find, even to skilled developers. To really understand what they are (and aren’t) requires hands-on, running-into-walls experience.
Docker for Mac, then, is a really great tool to start quickly and get your hands dirty. It hides most of the intricate detail of running a Linux virtual machine just to use Docker.
However, one of the pain points of Docker for Mac has been its shared
filesystem performance. Their custom solution
osxfs is excellent in terms of
features, but unfortunately very sluggish for, e.g., PHP applications. Long
forum threads have been dedicated to this issue.
Setting up a new development machine gave me the opportunity to try and fix the issue properly.
Other people have already tinkered with what’s actually inside the virtual machine that Docker for Mac so carefully hides. In his article “Meet Moby in Docker for Mac”, Luc Juggery explains how to get shell access to the machine. Kenn Herman has already created a tool d4m-nfs that automates logging into the machine and setting up an NFS share.
One of the perks of how Docker for Mac works, is that its virtual machine boots from a ramdisk, which means changes to the system do not persist across restarts. Side-effect of this is that something like d4m-nfs has to rerun everytime Docker for Mac has started.
I wanted something (subjectively) neater, a solution that’d also work with containers that start automatically.
The end result is in my docker-for-mac-nfs repository on GitHub, but I’ll describe the process below.
The boot scripts mount a persistent filesystem
/var, which is backed by a
virtual harddisk. If any part of the boot process had executed something from
there, we would be able to create a persistent hook, that’d even last across
Docker for Mac upgrades. Unfortunately, it seems there’s nothing like that.
So on to plan B, which was to modify the ramdisk. The image for the ramdisk is
located in the application bundle, typically at
/Applications/Docker.app/Contents/Resources/moby/initrd.img. This is a
regular ramdisk file for Linux to load, in compressed CPIO format (as revealed
To get NFS running in Moby, we first need to add the
nfs-utils package, which
chrooting into the ramdisk and running
apk. We can actually use
Docker to do this!
As it turns out, macOS ships with
cpio utilities from libarchive,
which allows easy conversion between formats on the fly. The command below
creates a new (uncompressed) TAR-archive, from the ramdisk contents:
tar -cf initrd.tar @initrd.img
This is exactly the format
docker import wants as input. Instead of creating
an intermediate TAR-archive file, we can stream it directly into a new Docker
tar -c @initrd.img | docker import - dummy.example/moby-initrd:latest
Now we can start containers from this image and modify them in any way we want!
After we’re finished making modifications, exporting a container back to a compressed CPIO archive is as easy as reversing the process:
docker export my-container | tar -czf initrd.img --format newc @-
tar utilties actually work on all archive types
supported by libarchive, not just CPIO and TAR archives.)
My initial approach to actually making modifications was to leverage the
existing Docker tools and create a Dockerfile. But exporting a Docker image
(the result of building a Dockerfile) as a flat filesystem turns out to be
docker save dumps individual layers, instead of a flat image, and
docker export only works on containers, not images.
So instead, I went for simply creating a container, copying in some files, and running a small script.
First, the simplest modification we need to make, installing
apk update apk add nfs-utils
Now for actually mounting the NFS share.
From the shell of a running Moby virtual machine, mounting the share was fairly easy:
mount -t nfs -o noacl,noatime,nolock,async 192.168.65.1:/Users /Users
Some observations trying various invocations of
mount with different options:
When the virtual machine talks to your Mac host, the host sees traffic from
localhost. Something to keep in mind when setting up your NFS exports.
You probably also want to map ownership of files on the share to your regular user on your Mac. This is what my
/Users -mapall=501:20 localhost
You need to start the
rpcbindservice in the virtual machine before NFS shares work. You can do so with:
Your Mac needs the following setting in
nfs.server.mount.require_resv_port = 0
My best guess is this is a quirk of the networking setup in Docker for Mac.
udptransport doesn’t work. Perhaps also a networking quirk, but UDP traffic sent to the Mac host seems to not arrive at all.
I’ve always specified
asyncfor shares, so wouldn’t know the exact performance impact. But
nolockapparently makes a big difference.
With this knowledge, and manually mounting the share working, I now wanted to automate this in the boot process.
All my attempts at adding the share to
/etc/fstab in the ramdisk failed, with
the connection seemingly timing out. Digging through the boot scripts, it looks
like this should work, with them properly waiting for the network before trying
to mount NFS shares. But I eventually stopped trying and moved on.
Instead, the mount is now setup using a boot script
and added to the default runlevel by creating a symlink
The boot scripts are in OpenRC format, but plenty of examples are already in
the ramdisk, so referencing documentation is not really necessary. I simply
specified that the
usermount ‘service’ needs the
rpcbind service, and
should run before the
With that, build the new ramdisk, replace it in the application bundle, and restart Docker for Mac!
The final product can be found in the docker-for-mac-nfs GitHub repository.
It contains an
import.sh script to create the base Docker image from the
stock ramdisk, which you’ll need to do just once. After that, run
get a new image derived from the base, with modifications.
This way, you can quickly iterate on modifications by repeatedly running
make.sh and copying the image to the application bundle. (You should probably
keep a backup of the original ramdisk around somewhere!)
Though if you’re going to use this, note that new versions of Docker for Mac will require rerunning all steps. Upgrades will likely include a new ramdisk, which may also be incompatible with the modifications.