Maybe I just haven't been paying attention before, but in the last year or so, there seems to have been an increase in discussion around the Nix package manager. This is also what got me to finally try it, and I'm very happy with the results, even if the road is a little bumpy.
I don't believe I'm the only one to mention the documentation is lacking a little finish. What I personally find is that there's just a lot Nix allows you to do, and no two people do things the same way. On top of this, the official Nix manual may try to teach you a whole bunch of things you won't be using in practice.
This post is an attempt to help smooth the road a bit. What I will try to do is go through Nix terminology bottom-up, and hopefully explain concepts that way. But this will be very much from my perspective, and will contain some opinion as well.
A little bit of what I've accomplished with Nix so far:
Setup the NixOS server hosting the publicly accessible applications of the Portier project, which includes packaging Rust and Python tools and applications. The configuration for this is public and can be found in the portier/public-infra repository on GitHub.
Setup my own MacBook using nix-darwin and home-manager, replacing my private dotfiles repo, vimplug, oh-my-zsh, and almost everything I did with Homebrew before. (I still use homebrew-cask.)
But, I still consider myself quite new, so do keep in mind there are probably mistakes and inaccuracies here!
Now, let's get started.
Nix
Nix is often called a package manager, because like other package managers, its intended purposes is to manage installations of all the tools and applications you need. You'll find Nix stretches this definition a bit, and is used for both smaller and larger tasks than just managing package installation.
Part of Nix itself are the Nix expression language, and a set of command-line tools.
Nix expression language
A functional programming language with lazy evaluation.
This is the language used to describe package builds, and it contains some unique language constructs to make that work. Also important to know is that all values in the language are immutable, and the language itself does not allow side-effects (all code and functions are 'pure').
I'd personally argue that working with Nix as a user is 80% doing interesting things with the Nix expression language, and 20% package management.
I'll refer to this as 'the Nix language' from here on.
Nix expression
Literally an expression in the Nix language, but a term very often used in passing.
In the Nix language, expressions are the top-level language construct. (As opposed to, for example, statements in JavaScript / Python, or definitions in C and friends.)
The term 'Nix expression' is also sometimes used to describe files containing them, with the .nix suffix. Notably, .nix files are not referred to as 'libraries' or 'modules', but more often as 'imports', after the built-in function used to load them. (For example: "Import the Nix expression from
foo.nix.")
Attribute set
A set of name/value pairs. Very similar to a dictionary in Python or an object in JavaScript.
{ foo = 3; "bar.baz" = 5; }
While this is an incredibly basic concept, I felt like I needed to highlight the term for it specifically, because it can be easily missed when skimming ahead through manuals.
Derivation
A special type of value in the Nix language to describe, essentially, a build step.
Derivations are a subtype of an attribute set, and created using the language built-in function derivation. (Though, you'll often end up using a wrapper for it.)
A derivation takes some inputs and produces some outputs. Because of lazy evaluation, the derivation is only actually built when one of the outputs is evaluated in the Nix language. Of course, this will also cause inputs to be evaluated, which may in turn be more derivations. This is how packages depend on eachother in Nix.
The most common scenario when packaging for Nix is for a derivation to have a single output called out (the default), and to use that as the installation prefix for whatever tool you are packaging. So, executables would end up in
$out/bin, for example.
It is also important to note that derivations are not just used to build packages, but anything really, like configuration files for example.
Derivations are really the core concept that makes Nix tick.
Derivation builder
One of the required inputs of a derivation is the builder, responsible for actually producing the outputs. This is often simply a shell script.
Here is one of the simplest possible derivations:
derivation {
name = "hello-world";
system = "x86_64-linux";
outputs = [ "out" ]; # This is the default, and can be omitted.
builder = "${pkgs.bash}/bin/bash";
args = [ "-c" "echo 'Hello world!' > $out" ];
}
The builder is assigned some output paths on the filesystem, one for each output declared. Here, we only have out, and the path for that output is then made available in the $out environment variable. The builder can create a directory there, but may also just write a single file.
Fixed-output derivation
A derivation for which the output is known ahead of time, via a content hash.
One if the basic steps in packaging is to download the source code of whatever tool you are packaging. In Nix these steps are just more derivations, created by wrapper functions like fetchurl.
Simply fetching a URL without any verification is a bad idea, because the contents of the URL may change, breaking the immutable builds Nix tries to achieve. Not to mention: a package manager that doesn't verify downloads would be a huge security risk.
Nix solves this by adding a content hash, which is what 'fixed-output' refers to. This looks like:
pkgs.fetchurl {
url = "https://...";
sha256 = "123...";
}
(This calls the function fetchurl with an attribute set as its argument. The return value of fetchurl is a derivation.)
Nix store
This is the filesystem storage for derivation outputs, at /nix/store.
So we've established derivations produce output on the filesystem, but these outputs are also immutable values in the Nix language. These values are simply string paths in the format:
/nix/store/<HASH>-<NAME>
Nix determines these paths once all inputs have been evaluated, and before
the builder even starts running. In fact, the $out variable in the builder script above is already the final /nix/store path.
The hash in the path is a content hash of all inputs, once evaluated. This means the hash encompasses not just what dependencies and versions are used, but the specific build of all those dependencies. It changes even if only the compiler used to build your package had a minor version bump, for example.
Whenever possible, packages refer to dependencies using the full /nix/store
paths. Nix actually goes through lengths to patch various parts of upstream tools to apply this everywhere. For example, you can see this with shared libraries by running something like ldd $(which bash) on a NixOS system:
linux-vdso.so.1 (0x00007fffc547e000)
libreadline.so.7 => /nix/store/ms1ris36xzyx9rzyss4h7pir759adc2d-readline-7.0p5/lib/libreadline.so.7 (0x000071e760ba4000)
libhistory.so.7 => /nix/store/ms1ris36xzyx9rzyss4h7pir759adc2d-readline-7.0p5/lib/libhistory.so.7 (0x000071e760b97000)
libncursesw.so.6 => /nix/store/kpw4kmc74djprg3bjc5rxblij46jdmnf-ncurses-6.1-20190112/lib/libncursesw.so.6 (0x000071e760b26000)
libdl.so.2 => /nix/store/9hy6c2hv8lcwc6clnc1p2jf09cs5q9dp-glibc-2.30/lib/libdl.so.2 (0x000071e760b21000)
libc.so.6 => /nix/store/9hy6c2hv8lcwc6clnc1p2jf09cs5q9dp-glibc-2.30/lib/libc.so.6 (0x000071e760962000)
/nix/store/9hy6c2hv8lcwc6clnc1p2jf09cs5q9dp-glibc-2.30/lib/ld-linux-x86-64.so.2 => /nix/store/9hy6c2hv8lcwc6clnc1p2jf09cs5q9dp-glibc-2.30/lib64/ld-linux-x86-64.so.2 (0x000071e760bf4000)
And remember the builder example above?
builder = "${pkgs.bash}/bin/bash";
Once evaluated, that also refers to a very specific build of Bash somewhere in
/nix/store.
This match between immutable values in the language, and immutable files in
/nix/store, is what allows Nix to precisely describe a very specific build of a package or even complete system, encompassing all its dependencies. It also naturally provides reproducable builds.
Binary cache
A cache of derivation outputs. Comparable to binary packages in other package managers.
These simply host prebuilt derivation outputs that Nix can download and copy straight to /nix/store. Especially useful for large, expensive to build packages like glibc or gcc, but you'll find probably more than 90% of what's packaged officially has a binary available.
Profiles
A set of packages to make available to the user, so they can be used normally from a shell. Also sometimes called 'user environments'.
Profiles can be managed with the nix-env tool included with Nix, but many Nix users opt for different solutions. (Like NixOS, nix-darwin, or home-manager, because they allow you to write out the desired packages declaratively, in a configuration file.)
A chain of symlinks is in place for each user:
$HOME/.nix-profile
/nix/var/nix/profiles/per-user/$USER/profile
/nix/var/nix/profiles/per-user/$USER/profile-42-link
/nix/store/<HASH>-user-environment
The user has $HOME/.nix-profile/bin in their shell PATH. The final
user-environment directory then contains a combined directory tree of all packages installed.
The numbered symlink encodes 'generations'. This way, it's easy to switch back to a previous state, simply by switching the symlink. (nix-env -G <NUM>)
Similarly, on NixOS you'll find there is a system profile with all packages installed and available system-wide / to all users.
Garbage collection
The process of removing unused files from /nix. Abbreviated to 'GC'.
As you keep using Nix, it will continue building up files in the store, and continue building up old generations of profiles. These need to be cleaned up via nix-store --gc or its wrapper nix-collect-garbage. This can be automated by simply running one of them at an interval (usually daily).
The process is actually quite simple: Nix scans the symlinks in /nix/var/nix
and deletes any store items no longer reachable through those symlinks. This directory contains the profiles subdirectory, but also a gcroots
subdirectory that can be used by tools built on top of Nix. All of these symlinks are called 'garbage collection roots' or 'GC roots'.
Channels
Collections of Nix expressions fetched from an external source. Comparable to a package registry / repository in other package managers.
Nix channels are managed by the nix-channel tool included with Nix. This tool simply downloads and extracts tarballs, which must contain a default.nix to be imported.
Very often, you'll find channels are pulled directly from a GitHub repository, using the special GitHub URL to download a tarball from the master branch. Even the official Nixpkgs channel can be used this way, though standard Nix installations use a nixos.org mirror.
The nix-channel tool reuses the profiles functionality. There's again a chain of symlinks for each user:
$HOME/.nix-defexpr/channels
/nix/var/nix/profiles/per-user/$USER/channels
/nix/var/nix/profiles/per-user/$USER/channels-42-link
/nix/store/<HASH>-user-environment
In the final user-environment directory, you'll find symlinks for each channel by name, pointing to their extracted contents.
NIX_PATH
Like PATH in a shell, but for Nix language imports. A semi-colon seperated list of search paths.
By default, NIX_PATH contains at least
/nix/var/nix/profiles/per-user/root/channels, which means channels registered by root are available to everyone. (This way, the root user is responsible for managing the main copy of Nixpkgs on the system.)
Elements of NIX_PATH are used to resolve angular imports, like:
import <nixpkgs>
Each directory in NIX_PATH is then searched in order for nixpkgs, and the first hit is used.
NIX_PATH also allows an element to describe a single import name directly with:
NIX_PATH="nixpkgs=/path/to/nixpkgs"
This can be used for any purpose, of course, but is most often used by systems like NixOS to encode the location of configuration:
NIX_PATH="nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels"
Nixpkgs
The official collection of Nix packages. Fresh Nix installations will have a channel setup for Nixpkgs, usually in the root user account.
When manually imported, you get a factory function that takes an attribute set containing optional configuration. This configuration is used to switch on/off non-free packages, for example. The return value of that function is an attribute set containing all of the available packages. (Lazy evaluation makes this cheap.)
A simple, manual import then looks like:
import <nixpkgs> {}
But more often than not, you don't have to import Nixpkgs manually, because it is configured elsewhere. Look for an argument or variable called pkgs, which will contain an already-configured Nixpkgs.
Nixpkgs also functions as a standard library for the Nix language. The Nix language itself has some built-in functions, but Nixpkgs provides many useful extensions in the lib attribute.
Overlays and flakes
These are methods to customize Nixpkgs. Overlays is the currently used method, while flakes is new, and its implementation is still experimental at the time of writing.
In general, these allow adding new packages or customizing existing packages. The set of overlays to use are part of the configuration provided to the Nixpkgs factory function, but there are more ways to set this.
nix-daemon
A daemon process that manages /nix for multple users and sandboxes builds.
This daemon runs as root, but does not actually evaluate Nix language code. It does, however, perform the actual derivation builds. Even though builds are sandboxed, usually only a select group of users will be allowed to use Nix on a system.
NixOS
Already mentioned several times, NixOS is a complete Linux distribution on top of Nix.
NixOS takes a configuration.nix, and uses derivations to generate real configuration for a complete Linux system. Over-simplified, it can be seen as a large build step for /etc. This build step automatically pulls in any packages it needs to realize your system configuration, just like any other dependency.
Processing of configuration.nix is best seen as adding an entire new layer on top of Nix package builds, that introduces all new concepts. But this new layer can use and modify the 'lower' layer of package building.
On a NixOS system, large chunks of /etc/ are symlinks, also in a chain:
/etc/bashrc
/etc/static/bashrc
/nix/store/<HASH>-etc/etc/bashrc
Not visible here, because this chain skips a step, but this build in
/nix/store is actually part of the system profile. Because it uses a profile underneath, it has generations you can rollback to. (nixos-rebuild switch --rollback)
NixOS can also manage packages for individual users on the system. These work a bit like nix-env profiles, but live separately in /etc/profiles.
Modules
Nix expressions that modify and extend NixOS system configuration.
Crucially, packages in Nixpkgs are just builds of software, and don't include systemd unit files. That's where modules come in.
Modules are Nix expressions that are loaded while processing NixOS
configuration.nix, and which provide new options. Standard modules preloaded by NixOS often have an enable option to activate them, and custom modules can be loaded using the imports = [ ... ] list in configuration. (Not a language-level import.)
When enabled, modules may modify various parts of NixOS configuration alongside your own configuration.nix, such as creating /etc files and adding new systemd units. These often pull in packages as a consequence. For example, an Nginx module pulls in the Nginx package, and creates the necessary systemd service unit.
Many of the standard NixOS modules actually live in the same repository as Nixpkgs.
(You'll find home-manager and nix-darwin, discussed later, also use the modules concept, but that code does not live in Nixpkgs.)
Activation
Additional steps to perform when switching to a new configuration in NixOS.
So far, we've only discussed build-time concepts. The NixOS layer on top of Nix also adds a run-time element, because services need to be started/stopped, for example. NixOS uses the term 'activation' for all of the additional run-time stuff nixos-rebuild switch does when switching configuration.
One part of this is a script called literally activate, which can be found in toplevel directory of your system profile. This script is composed by NixOS and various modules that were enabled, through the system.activationScripts
configuration option. (A quick way to find the script is to use the
/run/current-system symlink, which also points to the current system
profile generation.)
Actually starting/stopping services is handled by some Perl code as part of NixOS tooling, though.
nix-darwin
The macOS counterpart to NixOS, though obviously not a complete installable operating system.
The structure of nix-darwin is similar to NixOS: uses a
darwin-configuration.nix, is composed of modules, uses a system profile, and has an activation step. (darwin-rebuild switch)
The modules in nix-darwin allow you to have it selectively take over parts of macOS configuration, add new launchd services, and of course install packages.
Note that to get up and running with nix-darwin, you first need to do a 'multi-user install' as described by the manual. But nix-darwin will afterwards take over management of this installation, which works far better. Though, you probably will stumble setting up Nix on macOS. Quick tips: figure out how to setup an APFS volume for /nix, turn off Spotlight indexing for that volume, and heed warnings from the installer about files in /etc that it didn't touch. (Those require special attention.)
home-manager
Like NixOS and nix-darwin, but for setting up just one user's home directory.
Again, the structure is similar to NixOS: uses a home.nix, is composed of modules, uses a home-manager profile, and has an activation step. (home-manager switch)
The modules in home-manager allow you to have it selectively take over configuration of tools like git, zsh, vim, firefox, just to name a few, but the list is long. Home-manager can also setup systemd user services for you, and of course install packages.
Home-manager has support for both Linux and macOS, though the macOS support is not as complete at the time of writing. Notably, many modules that start user services don't yet know about launchd.
Really useful on single-user systems is that you can embed your home-manager configuration inside your NixOS or nix-darwin configuration. This also combines activation, meaning you don't have to touch the home-manager command-line tool at all.
Stand-alone Nix installations
I wanted to have a section to talk about the (non-NixOS) installation methods mentioned in the official Nix manual, because they're a thing, but they're wonky. On macOS, you have to go through these to 'move up' to nix-darwin, and I believe on other Linux distributions they may be the only supported method.
One of these options is a single-user Nix install, which means there is no nix-daemon, and /nix is owned not by root but by a regular user. This is already recommended against by the manual itself, because Nix can't sandbox builds in this setup.
The other option is a multi-user install, which runs nix-daemon properly as
root. The way the installer does this is a bit 'one-off', though, at least on macOS. Once installed, actually upgrading Nix itself becomes a very manual and confusing process.
Closing words
There's a bunch of stuff I haven't touched here at all, like NixOps or remote builds. Mainly because I haven't used them myself yet at all, and I wonder if I ever will! But hopefully that also means this list is a more comfortable subset of things interesting to new users.
What I personally found really powerful is using NixOS for individual servers with custom applications. I was never able to figure out packaging on Debian, but NixOS packaging is comparatively easy, despite the language learning curve. Bypassing packaging on Debian always meant I was left wondering: "How do I restart my service automatically if unattended upgrades roll out a new version of an important dependency, like OpenSSL or Node.js?" Also, the systemd integration in NixOS is amazing, and makes custom chroots a breeze.
I think the biggest hurdle to Nix for me was really the macOS installation process. Maybe the same problem exists trying out Nix on other Linux distributions? I feel like a great first-time experience outside of NixOS may really help build more interest.
Specifically, I think nix-darwin should have a combined installer, and the stand-alone install methods should be written off as 'advanced'.
Another gripe I have is with per-user channel management. It's fine, but the default should be tailored to systems with just one user, because I feel like that'll be the majority of installations? Specifically, I moved the Nixpkgs channel to my own user on my Mac, instead of root. Having to switch to root
just to upgrade Nix and Nixpkgs is really annoying. Now, I don't sudo at all, and only darwin-rebuild switch ever asks for my password.
Still, I sincerely hope none of this has turned you away from Nix, because I am hyped. As far as package and system management goes, it is futuristic. Time spent learning Nix is time spent well, in my opinion!