Jul 3, 2026

Upgrading Yubikey FDE to the FIDO2 based FDE on NixOS

So, when upgrading my laptop to NixOS 26.05 I got this amazing message:

Failed assertions:

boot.initrd.luks.yubikeySupport is deprecated, and it is unsupported with systemd stage 1. Support will be removed in 26.11 along with scripted stage 1. Hardware keys in systemd stage 1 are supported with systemd-cryptsetup(8). To migrate, enroll a key in a LUKS slot with systemd-cryptenroll(1). Usually, systemd will automatically detect the configuration at runtime, but if necessary, configure the corresponding crypttab(5) options with boot.initrd.luks.devices.<name>.crypttabExtraOpts.

Note: After migrating to a new LUKS slot, the old LUKS slot used for the scripted stage 1 implementation should be removed, otherwise it could interfere with falling back to a passphrase prompt in the event the hardware key fails.

I... Don't like how it sounds.

When installing NixOS several years ago I used the Yubikey based Full Disk Encryption (FDE) on NixOS guide which utilizes YubiKey’s HMAC-SHA1 challenge response mode. I’ve used it with the second factor passphrase. Now my system tells me this is no longer supported, and I must migrate to FIDO2. Without giving me a proper guide for this. There is a FIDO2 based Full Disk Encryption (FDE) on NixOS page which describes how to set up FIDO2 based FDE from scratch, but it doesn’t describe the migration steps.

So I had to figure it out and write a guide myself.

Step 1: Backups

Poking around full disk encryption is scary. Make a full backup of all the data you need. Obviously, you need to save it on an external storage, encrypted. I’ve used the Pika Backup app, which is a GUI wrapper around Borg. Don’t forget to validate your backups (i.e. make sure it is possible to recover data from them).

Step 2: Obtaining LUKS key

On the surface, the migration process is quite simple. You need to add a LUKS slot for your FIDO2 thingy. Easy-peasy:

sudo systemd-cryptenroll --fido2-device=auto

It will ask for a LUKS passphrase. If you followed the old FDE guide you probably don’t have it, because your passphrase is a combination of the yubikey challenge response and your second factor password. You can’t just type that. So we need to obtain the LUKS key, somehow. Luckily, the old guide has an entire section dedicated to that!

KEY_LENGTH=512
ITERATIONS=1000000
SALT=$(head -n1 /boot/crypt-storage/default)
read -s USER_PASSPHRASE
CHALLENGE=$(echo -n $SALT | tr -d '\n' | openssl dgst -binary -sha512 | rbtohex)
RESPONSE="$(sudo ykchalresp -2 -x $CHALLENGE 2>/dev/null)"
LUKS_KEY="$(echo -n $USER_PASSPHRASE | pbkdf2-sha512 $(($KEY_LENGTH / 8)) $ITERATIONS $RESPONSE | rbtohex)"

But you can’t just run that. You need to define the rbtohex and hextorb functions first. Fine, no problems with that. You will also need to compile the pbkdf2-sha512 program. Wiki page has instructions for that, but if you try to compile it, you will stumble into the fact that pbkdf2-sha512 is no longer in nixpkgs 25.11. Yay.

Here’s what you need to do to get it working:

# First, enter the shell with required utils
nix-shell -p gcc yubikey-personalization openssl

# Next, define convert functions
# Yes, you can just type that into your terminal.
rbtohex() {
    ( od -An -vtx1 | tr -d ' \n' )
}

hextorb() {
    ( tr '[:lower:]' '[:upper:]' | sed -e 's/\([0-9A-F]\{2\}\)/\\\\\\x\1/gI'| xargs printf )
}

# Now, we need to compile pbbdf2-sha512.
# Go to some folder where we will download its sources and compile it

# nixpkgs-24.05 still has it, so just grab it from there
curl -sL https://raw.githubusercontent.com/NixOS/nixpkgs/nixos-24.05/nixos/modules/system/boot/pbkdf2-sha512.c -o pbkdf2-sha512.c
# Now compile it against nixpkgs' openssl
cc -O3 \
  -I$(nix-build "<nixpkgs>" --no-build-output -A openssl.dev)/include \
  -L$(nix-build "<nixpkgs>" --no-build-output -A openssl.out)/lib \
  pbkdf2-sha512.c \
  -o ./pbkdf2-sha512 -lcrypto
# runtime lib path so it can find libcrypto.so
export LD_LIBRARY_PATH=$(nix-build "<nixpkgs>" --no-build-output -A openssl.out)/lib

# You will have a ./pbkdf2-sha512 binary now.

# Now let's extract the LUKS_KEY itself.
# These are default settings. Change them if you changed them during initial setup.
KEY_LENGTH=512
ITERATIONS=1000000
SALT=$(head -n1 /boot/crypt-storage/default)
read -s USER_PASSPHRASE
CHALLENGE=$(echo -n $SALT | tr -d '\n' | openssl dgst -binary -sha512 | rbtohex)
RESPONSE="$(sudo ykchalresp -2 -x $CHALLENGE 2>/dev/null)"
LUKS_KEY="$(echo -n $USER_PASSPHRASE | ./pbkdf2-sha512 $(($KEY_LENGTH / 8)) $ITERATIONS $RESPONSE | rbtohex)"

You will now have LUKS_KEY with your master key.

Step 2: Enrolling keys

We can’t just pass the master key as a passphrase. systemd-cryptenroll can accept the master key as a key file with --unlock-key-file option, but we really don’t want our master key to touch the disk, ever.

So first let’s make a small partition inside the RAM:

sudo mkdir -p /run/luks-migrate
sudo mount -t tmpfs -o size=1M tmpfs /run/luks-migrate

You can now write the key into file like this:

echo -n "$LUKS_KEY" | hextorb > /run/luks-migrate/luks.key

Great. Now we can use systemd-cryptenroll. Other thing you should check is where your LUKS is. Here’s an example from my machine:

❯ lsblk -o NAME,SIZE,FSTYPE,MOUNTPOINT
NAME                      SIZE FSTYPE      MOUNTPOINT
nvme0n1                 953.9G
├─nvme0n1p1               300M vfat        /boot
└─nvme0n1p2             953.6G crypto_LUKS
  └─nixos-enc           953.6G LVM2_member
    ├─partitions-swap       6G swap        [SWAP]
    └─partitions-fsroot 947.6G btrfs       /var/lib/docker

nvme0n1p2 is my LUKS container, and I will use it in the commands below.

Before enrolling my YubiKey FIDO2, I decided to also enroll a recovery key. You can do it like that:

sudo systemd-cryptenroll --unlock-key-file=/run/luks-migrate/luks.key --recovery-key /dev/nvme0n1p2

It will output a recovery key, with a QR code. Save it in a safe place, it will allow you to decrypt your disk without your Yubikey.

It’s not necessary to make a recovery key. I just did it as an escape hatch in case something goes wrong. You can delete LUKS slots later, I guess.

Now, enroll your Yubikey:

sudo systemd-cryptenroll --unlock-key-file=/run/luks-migrate/luks.key --fido2-device=auto --fido2-with-client-pin=true /dev/nvme0n1p2

It will ask for your YubiKey’s PIN code. Yep, you will have to enter your PIN code instead of password now. If you used single factor before, you can replicate that by setting --fido2-with-client-pin=false instead.

Step 3: Updating NixOS configuration

My setup looked like this:

{ config, lib, pkgs, modulesPath, ... }:

{
  boot = {
    initrd = {
      availableKernelModules = [ "xhci_pci" "thunderbolt" "nvme" "usb_storage" "usbhid" "sd_mod" ];
      kernelModules = [ "dm-snapshot" "vfat" "nls_cp437" "nls_iso8859-1" "usbhid" ];
      luks.yubikeySupport = true;
      luks.devices = {
        "nixos-enc" = {
          device = "/dev/nvme0n1p2";
          preLVM = true;
          yubikey = {
            slot = 2;
            twoFactor = true;
            storage = {
              device = "/dev/nvme0n1p1";
            };
          };
        };
      };
    };
    kernelModules = [ "kvm-intel" ];
    extraModulePackages = [ ];
    kernel.sysctl = { "vm.swappiness" = 5;};
  };
}

Several changes are needed:

Remove the boot.initrd.luks.yubikeySupport = true;
Remove entire boot.initrd.luks.devices.<device name>.yubikey section
Add boot.initrd.systemd.enable = true;
Set boot.initrd.luks.fido2Support = false;
Set boot.initrd.luks.devices.<device name>.crypttabExtraOpts = [ "fido2-device=auto" ];

Here’s how my config looks like after the changes:

{ config, lib, pkgs, modulesPath, ... }:

{
  boot = {
  initrd = {
    availableKernelModules = [ "xhci_pci" "thunderbolt" "nvme" "usb_storage" "usbhid" "sd_mod" ];
    kernelModules = [ "dm-snapshot" "vfat" "nls_cp437" "nls_iso8859-1" "usbhid" ];

    systemd.enable = true;
    luks.fido2Support = false;
    luks.devices = {
      "nixos-enc" = {
        device = "/dev/nvme0n1p2";
        preLVM = true;
        crypttabExtraOpts = [ "fido2-device=auto" ];
      };
    };
  };
  kernelModules = [ "kvm-intel" ];
  extraModulePackages = [ ];
  kernel.sysctl = { "vm.swappiness" = 5;};
}

Now, sudo nixos-rebuild boot and you should be done. Good luck. Make sure you have backups before rebooting.