My biggest problems with NixOS

Probably a year ago today, I learned about the magic of Nix and it’s Linux distribution NixOS. In theory, NixOS did/does everything I’ve ever wanted from a Linux distribution. I say in theory because unfortunately, NixOS suffers from a very common problem within the tech world, what I’m going to call the “Day Two” problem.

The “Day Two” problem, as I’m calling it here (though I’m sure it has another name) is when you’ve gotten past the “Hello World” phase of a new technology - you’ve gotten the tech demo working and you’re excited, - but then you have the uphill battle of learning the skeletons in the closet.

One beautiful feature of NixOS is that your entire operating system is declared in a single file called configuration.nix and in theory, this means all you need to completely replicate your installation on another machine is 1) a USB drive with the NixOS installer, 2) A copy of your configuration.nix leads me to the first problem:

What version of Nix is it anyway?

One really nice thing about NixOS is that the entire OS is defined as a tree of files, called a channel. For example here is NixOS 16.09

While this is amazing, it leads to the first problem which is that the configuration is only reproducible if you have the same channel checked out across all your machines, and unfortunately Nix doesn’t give you a way to pin or define the channel in your configuration file, and so you need to hope/pray you have the same version checked out if you want your build to be the same.

For comparison, docker avoids this problem by having a FROM stanza at the top of the Dockerfile. This gives the user (and of course the docker daemon) context for the commands that come after it. For example, apt install nginx only makes sense when run on a Debian derived distribution as well, the version of nginx you’ll get depends on which version of the Debian distribution you started from. Nginx on Debian 7 and Debian 8 would be different.

I would really like to see the NixOS team implement a way to manage nix channels directly from the configuration.nix file to avoid this problem. This would also make upgrades as easy as changing NIXOS_VERSION from ‘15.09’ to ‘16.09’ and running nixos-build

OK, fine, so you downloaded the same NixOS iso on all your machines, and you’re ready to install your operating system. If you’ve never installed NixOS before it’s also really nice and pretty straight forward. Basically, it works like this

Boot the installer,
Create a boot (/boot) and root (/) partition
Create a file called configuration.nix in /etc/nixos with your desired bootloader set
Run nixos-build

That’s it! It’ll automatically build and install all of the required packages to / and the desired bootloader to /boot and after a reboot it should all just work.

But unfortunately this leads me to my second “Day Two” problem with NixOS:

No way to declaratively manage state.

The only way to configure your partitions in NixOS is imperatively, namely,fdisk mkfs, vgcreate, lvcreate etc.

Ideally, Nix should have a partitions.nix file or similar that allows me to declaratively define a 200MB fat32 volume called /boot and the rest of the drive is a LVM volume group with a logical volume called / formatted as ext4. It could be as simple as

system.partitions = [
  '/boot' = {
    size = 200MB
    fs = "fat32"
  },
  '/' = {
    # no size, whatever is left over will be used
    type = 'lvm'
    fs = "ext4"
  }
];

Maybe this isn’t the job of the package manager, but for me, this would be a huge step forward in the usability of NixOS. I see this as being a great replacement for something like cloud-init. Imagine I have an AWS Launch Configuration for my web servers, they each have a 20GB EBS volume, 4GB ram, 4 CPU cores. It should be as easy as defining a NixOS AMI + pointing the LC to a configuration.nix file in an S3 bucket to configuration the whole instance.

But again, not a big deal right? Nix’s package format is actually a programming language so if I really wanted I probably could probably write a definition that does that.

So my partitions are set up and I’ve run the installer, and rebooted and now I’m in my user profile with all programs from my other machines installed right?

Unfortunately no, which is my third problem:

No declarative way to manage installed applications.

The prescribed way to install packages for a user is using nix-env -i <package-name>. Unlike NPM for example, there is no way to persist what’s installed. Unfortunately, there is no way to do a nix-env shrinkwrap and move all my packages to another machine.

“But wait” - I hear you say, as you bang away in the comments below - “You can define your packages in configuration.nix OR you can use nix-shell!”

Of course, non-root users don’t have access to the system configuration (for good reason!) so that’s not a good solution, but also, nix-shell its self is a Day Two problem with Nix - I need to know the nix programming language to use it.

Ideally, I’d like to see user nix packages managed the same way node_modules are managed, in which I mean that a user gets a ~/packages.nix or ~/profile.nix that gets updated when I run nix-env install <package-name> OR can be updated by simply adding a new package to the file and running nix-env install with no parameters (a la npm install)

Lifting Metadata out of the build function

So I do know a very tiny amount about the nix programming language (thanks mostly to @susanpotter on twitter, she’s my hero) and so far it seems that package metadata such as name and version are actually part of the “build function” instead of being passed into it.

For example, here is a snippet of a nix definition for installing plex:

in stdenv.mkDerivation rec {
  name = "plex-${version}";
  version = plexpkg.version;
  vsnHash = plexpkg.vsnHash;
  sha256 = plexpkg.sha256;

  src = fetchurl {
    url = "https://downloads.plex.tv/plex-media-server/${version}-${vsnHash}/plexmediaserver-${version}-${vsnHash}.x86_64.rpm";
    inherit sha256;
  };

  buildInputs = [ rpmextract glibc ];

  phases = [ "unpackPhase" "installPhase" "fixupPhase" "distPhase" ];

If you look at the src function you’ll note that the version of plex source that is fetched is based on the version variable, which would make me think that switching versions of plex would be as simple as changing that variable right?

Well, I’ve asked all the NixOS users I know (which to be fair, is only 2 or 3) and so far the answers I’ve gotten are

Redefine the plex function in your config with the version you want
Fork your channel and edit the plex definition

I went with the first option because it seemed like the simpler choice and now I can upgrade plex simply by changing the version in my configuration but this brings me to my last complaint:

Overriding definitions is difficult

One feature I really like in newer releases of Systemd is that you can override parts of a service definition, or you can full out replace it.

Better yet, this can be done by running systemctl edit service-name or systemctl edit --full service-name.

The former will create a file whose values override the values in the service, while the latter will give you a complete copy of the original file that you can edit however you want. Better yet, the original is left untouched, thus you can revert your change by simply deleting the service-name.overrides file that’s created.

I’d love it if this Nix team did the same for the nixpkgs. Changing the version of a package would then be a matter of nix-pkg edit plex << version = 1.9 (or something) nixos-rebuild. Done.

The future is bright

I want to stress that I don’t think NixOS is “doomed” or “broken” or anything like that. These are all minor issues that can and probably will get fixed making NixOS approachable for users of all backgrounds and skill levels.

Other than these complaints I really enjoy using Nix and I think it as a bright future as a package manage, an OS and (hopefully) as a replacement/augmentation for container build systems.

To me, functional package managers are the best solution to software packaging we have, and combined with containers and unikernels I think they’ll define the next generation of package mangement.

Finally, I hope I’m wrong about most of what I said and that someone will come along and set me straight :)