Nonchalant Guidance

Added on: Saturday, 27 May, 2023 | Updated on: -

Migrating to NixOS (Part 2)

Erase Your Darlings

In the last post, we covered how I ended up doing a NixOS installation to enable greater reproducibility of my system and packages. I hinted at my future plans to extend this reproducibility to implement “erase your darlings”, in which you explicitly define what files and directories you want saved and your entire system is reconstructed from scratch by NixOS using these saved files. Anything not in these specific folders and files is erased.

There are many reasons why you’d want to do this:

Avoid configuration drift: Instead of doing a super-important change in an obscure and hard-to-remember configuration file, this forces you to explicitly save that file in your NixOS configuration, or better, define that change in configuration.nix
Better reproducibility; by having a list of files required for boot, one could easily recreate the same install and quickly copy over Wi-Fi passwords, SSH keys, custom configs, VPN private keys, etc
Why just /, erase /home too?: if you really want to take this to the extreme, you can do this for your user’s home folder as well, persisting their user configs as well

Basic Setup

Erase Your Darlings requires 2 things:

A way to save and link saved files: for this we will use Impermanence
A way to erase the system, saving only /boot, /nix (NixOS requires these two to be present to boot) and whatever directory you use to save your files in (I used /persist). The original post used ZFS snapshots, while a guide I followed used btrfs snapshots, but in the end the simplicity of mounting / to tmpfs (basically a filesystem in RAM, so it will never persist) was just too much to resist.

Step 1: Installation and Prep

I originally tried to implement these changes in my old install, but after some difficulties (ultimately my mistake to not have /nix be a btrfs subvolume killed this entire attempt), I opted to do a fresh install. This was also a blessing in disguise, since I found that certain packages I’d installed using nix-env were highlighted to me by their absense, so I added them into the config.

I had to modify the installation a lot, and in the end I used the KDE live USB of NixOS to just do a manual install myself, bypassing the graphical Calamares installer.

Roughly speaking, here are the steps to install:

Create partitions
Encrypt relevant partition(s)
Mount those partitions

I decided to do a similar setup to my old install: a /boot/efi mountpoint for ESP, and a LUKS volume with btrfs on top. I ended up fixing my old script in my nix-config repo (GitHub mirror) and using that to automate the install.

The script also created the subvolumes required: /nix, /home, /persist

Copy over configuration.nix

I first checked hardware-configuration.nix to see what the setup is. To my delight, the subvolumes I’d created had been picked up by NixOS, so all I really had to do was tell NixOS to mount / on tmpfs:

  fileSystems."/" =
    { device = "none";
      fsType = "tmpfs";
      options = [ "defaults" "size=1G" "mode=755" ];
    };

Note: if you’re worried that I am just losing 1 GB RAM to /, don’t. The 1G size is a limit on the size, and is not kept reserved explicity for tmpfs. So, that means that I have only ever used at most 150MB of RAM mounting / to tmpfs, and that was after I’d left my machine on for 3-4 days. A fresh boot brings that number down to low double digits.

The subvolumes are mounted as follows:


  fileSystems."/home" =
    { device = "/dev/mapper/cryptroot";
      fsType = "btrfs";
      options = [ "compress=zstd" "noatime" "subvol=home" ];
    };

  fileSystems."/nix" =
    { device = "/dev/mapper/cryptroot";
      fsType = "btrfs";
      options = [ "compress=zstd" "noatime" "subvol=nix" ];
    };

  fileSystems."/persist" =
    { device = "/dev/mapper/cryptroot";
      fsType = "btrfs";
      options = [ "compress=zstd" "noatime" "subvol=persist" ];
      neededForBoot = true;
    };

Note that it also detected the device name I’d set for the LUKS partition in my shell script. Yay!

Run nixos-install

This was the most time consuming process since it downloaded everything defined in my config, which had by then become quite sizeable.

Reboot and pray it works

Impermanence

After booting in and verifiying everything worked, my attention now turned to persisting relevant data.

The NixOS wiki page on Impermanence has a good starting point for this. I ended up following its advice at first and specialized which folders I wanted to save later down the line based on what programs I was using that needed that persistence.

I don’t have much Impermanence code for my home directory, and opted to remove home-manager entirely. I didn’t really like how it worked (read-only linking), and Impermanence would do the job better. I only link a couple important folders and configs for now, but plan on expanding this number.

Quirks

Home-Manager didn’t seem to work after I copied over my backup. I tried reinstalling and a couple other steps, but it just wouldn’t take. Oh well, not a huge loss.
The KDE display scaling always defaults to 125% when I login. I haven’t been able to track down what config in / is responsible for this (I don’t think it is /home because it would’ve persisted otherwise)
/home is a mess. I would really like to separate out dotfiles and personal data (or at least just dotfiles), but because there are so many it will take me some time to do this.

Conclusion

Erase Your Darlings and Impermanence haven’t really changed how I use my system, but they gotten me much closer to ensuring that if my machine burns down, my Internet connection and shipping speed will be the main bottlenecks in getting me back up and running again.

Before NixOS, I could get personal data and user dotfiles restored. With my previous NixOS install, I could get programs installed again, With my latest install, I can also get some vital system configs back.

This was also an opportunity to reset and rebuild. There are (ironically) some steps I’ve left out in this journey which I don’t remember (like Impermanence not persisting stuff in my home folder correctly, so I had to change permissions on that folder manually for my user as root) that will surface when I do another clean install. Till then, this is good enough for me, and the next time, I’ll test this entire stack by doing an install from scratch using my repo and see how close I get to reproducing the old system with as little manual work as possible.

This website was made using Markdown, Pandoc, and a custom program to automatically add headers and footers (including this one) to any document that’s published here.