64. The journal's integrity check depends on the code that reads it

Status: accepted. Half of it was a guard rather than a fix — said in the decision rather than left to be discovered — and the fix it names has since been made in ADR-0067, without moving a digest. Concerns the audit sink ADR-0010 names as an invariant and ADR-0060 seals.

Context

The journal is one line of JSON per record, each naming the hash of the one before it. A record’s hash covers its sequence number, its timestamp, the previous hash, and the entry. Editing a record, reordering two, or removing one from the middle all break a link, which is what the chain is for.

The digest over the entry is taken like this:

let body = serde_json::to_vec(entry)?;

entry there is a &Entry — a typed Rust value, parsed out of the line a moment earlier. Verification does the same: it reads the line, deserialises it into an Entry, serialises that back, and compares the hash. So the check does not ask what is on the disk. It asks what this binary makes of what is on the disk, and those are the same thing only for as long as the type is frozen.

They stop being the same thing on any of these:

• a field added to a variant, even with #[serde(default)] — the key appears where it was absent;
• two fields swapped, because serde emits a struct’s fields in declaration order;
• a field or a variant renamed;
• any change to what Entry embeds: Principal, Classification, Verdict, TransactionId, SessionId, and everything under those.

The last one is the dangerous one, because somebody adding a field to a principal is not thinking about a journal.

What follows is not a wrong answer, it is a machine that will not start. verify reports JournalError::Altered — “record 4 has been altered” — the keeper treats a journal that does not verify as fatal and returns FAILURE without binding its socket, and tacidd requires the keeper. The machine boots with no nervous system, and it gets there by following its channel, which is the one road this whole system is built to travel. Boot counting abandons the generation after three boots, so the immune system limits the damage; that is not the same as the damage not happening, and the rollback would land on every machine on the channel at once.

Nothing in the tree would have caught it. Every machine test writes its journal with the same binary that verifies it, so the failure needs a journal written by an older version, and there was nowhere one existed. The one test that sounded like it covered this — hashing_is_stable_ across_runs — hashes the same value twice in the same process and asserts the two agree, which is determinism and not stability.

Decision

1. The record format is frozen, and there is now a place where it is frozen. the_record_format_is_frozen_and_this_is_where_it_is_frozen pins five records to literal digests: a note, an egress line, a secret handle, a ruling that embeds a Verdict and an IrreversibleReason, and an approval that embeds a Principal. Any change that moves the bytes fails it.

2. Its failure message says what the failure means and what not to do about it. That a journal already written now fails to verify; that a keeper which cannot verify its journal does not start; and that replacing the numbers with the new ones hides the failure rather than fixing it, because the journals on other people’s machines still hold the old bytes.

3. Changing the format is a migration, decided deliberately, and not a refactor. The test does not forbid the change. It makes it impossible to make by accident.

4. The real fix is named and not yet made. (Made since, in ADR-0067. The paragraph stands as it was written, and the digests did not move.) The hash should cover the bytes as they were written — Record.entry as a Box<RawValue>, compute_hash over those bytes, a typed accessor for readers — so that the integrity of the record does not depend on the types of the program checking it. That is the property a tamper-evidence mechanism ought to have, and it is backward compatible by construction: a record written today has raw text identical to what to_vec produces, so the digests pinned above must not move when it lands. If they move, the change is wrong, which makes the guard above the test for the fix as well.

Consequences

• The fragility is enforced rather than removed. That is a real difference and the reason it is written down this way: somebody reading only the decision should not come away thinking the journal is now robust against a change to Entry. It is robust against an accidental one.
• Any future change to a type in tacid-protocol that Entry embeds now has a test in another crate that fails, with an explanation. That is the coupling made visible; it was there all along, and nothing said so.
• The five fixtures are a sample, not a proof. A variant nothing pins can still be changed silently. Pinning every variant would be better and is the fix’s job, not the guard’s.
• One field of a record the hash does not cover: its own id. It is minted from the sequence number and the timestamp, both of which are covered, so it carries nothing of its own and need not be hashed — but nothing was checking that it agreed with them either, so it could be rewritten to anything and every digest would still hold. verify re-derives it now, which closes the gap without moving a single byte of the format: a field inside an integrity check that the integrity check does not reach is a field somebody can write anything into.

Alternatives rejected

Leave it and fix Entry carefully. “Be careful” is not a mechanism, and the people who will change Principal in a year are not the people who read this file.

A version number on each record, with a hasher per version. It works and it means keeping every past serialisation of every past type alive in the tree forever. The bytes on the disk are already the version.

Verify only the links and not each record’s own hash. It would make the format irrelevant and throw away the thing the chain is for: a record edited in place keeps its position and its neighbours, and only its own digest catches it.