0090. A generation writes down what its initrd carries

Status: accepted. Follows ADR-0089, which put a boot screen on every machine with a screen, and answers the failure that screen has when nobody is looking.

Context

Plymouth draws on a DRM device. A DRM device exists in stage 1 only if its driver was in the initrd, because nothing inside an initrd loads a module that is not in the archive. So a machine whose display driver arrives in stage 2 has no picture for the whole of its boot — including the prompt for the passphrase to a root that Tacid encrypts by default (ADR-0012) — and shows the console instead.

It fails silently, which is the part that matters. Every unit succeeds. systemd-modules-load inserts the driver on time, for stage 2. Plymouth starts, finds no device, picks whatever renderer it can and keeps it, and exits at the end of the boot having drawn nothing. There is no failed unit, no warning, and no line in the journal that says the picture had nowhere to go.

That is how it was found, and it took five runs to see. checks.*.desktop photographs the boot of an emulated machine and asserts the ground is the background token. It kept photographing #141a1f — the console’s colour, not the splash’s — and the first three explanations were all about the theme. It was not the theme. virtio_gpu loaded at 12.7 seconds, four seconds after the splash had already been replaced by a login prompt, and the frame was a picture of a virtual console that had never had anything drawn over it.

nixos-generate-config does not help here and is not meant to: the boot.initrd.availableKernelModules it writes are the storage and USB controllers needed to find a root filesystem, and a display adapter is not one of them. Nothing in nixpkgs’ plymouth module adds a DRM driver either — it adds plugins, themes, fonts and udev rules. So on a stock configuration with a splash, the common outcome is the failure above, and it is common because nobody is told.

The obvious check cannot be written, which is the reason this record exists. After switch-root the initrd is gone: it is a compressed archive in the store, nothing it contained is on the filesystem any more, and “which modules were in it” has no reader on a running machine. The information exists only where the system was evaluated.

Decision

A generation writes what it built into /etc/tacid-system/boot.json, beside services.json and declared.json, which are there for the same reason: a system’s own account of itself, readable off a store path, with no daemon running and nothing evaluated. It carries whether the system has a boot screen and the modules its initrd carries — boot.initrd.kernelModules and boot.initrd.availableKernelModules together, because both put a module in the archive and the difference between them is when it loads, not whether it is there. system.boot reports both fields, and reports them as null on a system that did not write them: an older generation and a machine with no boot screen are different answers.

tacid.hardware.gpu.profile puts the driver for the card it names into the initrd. A body that has said which card it has has said which module its screen needs; the profile enum already knows the seven answers, and each maps to a module — i915, amdgpu, nouveau, the three NVIDIA modules — except software, which names no card. availableKernelModules rather than kernelModules, so udev loads it when it sees the card rather than forcing it on a body that turns out not to have one: a profile is a declaration about hardware, and this respects the case where the declaration is wrong.

doctor boot reports the case the profile cannot cover. software is the default and is what the installer writes when it finds no discrete vendor, so the machine most likely to have this failure is exactly the one whose declaration says nothing — and for that machine the honest statement is that nothing was put in the initrd for a card, so if there is one, its driver arrives in stage 2. A note rather than a warning: a machine with genuinely no display adapter is correct to be there and is not broken.

Both halves of the comparison come from the generation, because the running machine cannot supply one of them. This is the part that was wrong twice before it was right.

The original design asked the machine which module was drawing and compared that against the initrd’s list. system.graphics gained a drm field for it, on the reasoning that the PCI driver is the display driver on a great many machines and is not on any virtual one — QEMU’s adapter is a PCI device whose driver is virtio-pci, the transport. That reasoning is correct. The read is not available. checks.*.desktop prints the tree it was reasoning about:

/sys/class/drm/card0 -> /sys/devices/pci0000:00/0000:00:08.0/drm/card0
  device        -> /sys/devices/pci0000:00/0000:00:08.0
  device/driver -> /sys/bus/pci/drivers/virtio-pci
/sys/bus/pci/devices/0000:00:08.0/ holds drm/, driver …/virtio-pci

The DRM node hangs off the PCI device itself; there is no child holding drm/; and no path from that card names virtio_gpu. Two implementations — one through /sys/class/drm/<card>/device, one through whichever directory holds drm/ — both returned virtio_pci, the transport with an underscore in it, which is the shape of a right answer and the content of a wrong one. Both passed their unit tests, because the fixtures encoded the same wrong model of sysfs. The leg caught both.

So the drm field is gone, and the comparison moved to where both halves are known. The generation writes gpu_profile — the card this body declared — and display_in_initrd: whether the driver that profile needs reached the archive, or null when the profile named no card, because “this body did not say” is a third answer and not a no.

The finding says only what is certainly true. A driver that was not in the archive could not have drawn: that is a fact about how initrds work and it is what the warning says. A driver that was in the archive is a necessary condition and not a sufficient one — this project has a machine where virtio_gpu in the initrd still produced no picture, five times — so the passing finding says that the initrd carries the module and stops there. It does not promise a boot screen.

Consequences

A machine cannot be born signing, and the Secure Boot leg found out by trying. Bringing the same rigour to checks.*.secureboot meant giving its nodes a real EFI partition, which meant virtualisation.useBootLoader, which builds each node a disk image with its bootloader installed. The signing node’s bootloader is Lanzaboote, and Lanzaboote signs what it installs with keys the owner makes at runtime:

Installing Lanzaboote to "/boot"...
Failed to install generation 1: Failed to read public key from
/var/lib/sbctl/keys/db/db.pem: No such file or directory

That is not a limitation of the test framework. It is the order a Tacid machine reaches Secure Boot in — boot, keys, then the first generation whose bootloader is signed — showing up as a build failure, and it is worth knowing that the option on a machine with no keys yields a system that cannot install its bootloader rather than one that installs an unsigned one.

The check has three answers and the third is the important one. Skipped when the generation did not write its account, because “this machine has no boot screen” and “nobody looked” are different, and reporting the second as the first is a check that has quietly stopped checking. The same rule as secure_boot in the same domain, for the same reason.

The emulated machine is now a configured machine rather than a concession. checks.*.desktop puts virtio_gpu in its node’s initrd because that is what a machine which had declared its card would have, and asserts on the same machine that doctor boot reads it: that the file the module writes is the file the check reads, and that what it finds there is this machine’s real card. The failing case, the no-driver case and the generation-says-nothing case are unit tests, where they can be all four rather than whichever one a machine that boots happens to be in.

Nothing here is a promise that a boot screen appears. The emulated machine still has not shown one — virtio_gpu reaching its initrd changed the moment the driver loads and did not, over five runs, change the frame. That is a separate open question about this emulator, and it is exactly why the passing finding is worded as an observation about an archive.

Alternatives rejected

Reading the initrd. It is a file in the store and it could be unpacked. Unpacking fifty megabytes to answer one question, on every doctor boot and every health gate, to learn something the evaluation already knew, is the wrong trade twice over.

Reading the journal for when the driver loaded. The timestamps are there and the answer is in them. It would be a check that parses prose, which is the one thing every fact in this tree is written to avoid: a check that starts lying after somebody’s log format grows a field.

Putting every known DRM module in every initrd. It would work, and it would make the archive on a machine that declared its hardware carry six drivers for cards it does not have. A profile is a declaration; honouring it is the point of having one.

Warning from the module at evaluation time. The module cannot: whether a machine’s card is covered depends on what is in the machine, and evaluation has never seen it. This is the shape of question doctor exists for.