0067. A record is bytes, and what those bytes mean is a separate question

Status: accepted. Makes the fix ADR-0064 named and did not take. Concerns the audit sink ADR-0010 names as an invariant and ADR-0060 seals.

Context

ADR-0064 found that Record::compute_hash re-serialised a parsed Entry to produce the digest it checked a record against. So the digest was not over what was written; it was over what the current binary makes of what was written. Add a field to a variant, reorder two, rename one, or change anything Entry embeds, and every record already on disk stops matching its own hash — a keeper that cannot verify its journal does not start, tacidd requires the keeper, and the machine comes up with no nervous system, by the update path.

That ADR froze the format with a test carrying five digests and said, in as many words, that the guard was not the fix and that the fix was this:

The hash should cover the bytes as they were written — Record.entry as a Box<RawValue>, compute_hash over those bytes, a typed accessor for readers — so that the integrity of the record does not depend on the types of the program checking it.

Decision

A record holds the entry as it was written, and the digest is over those bytes.

pub struct Record {
    …
    /// What happened, exactly as it was written.
    #[serde(rename = "entry")]
    pub written: Box<RawValue>,
}

impl Record {
    /// What happened, read with today's understanding of what a record can say.
    pub fn entry(&self) -> Result<Entry, JournalError>;
}

The two names carry the distinction, because the distinction is the whole point: written is what is on the disk; entry is an interpretation of it, made by whichever build is asking. A field called entry holding raw bytes beside a method called entry() returning a parsed one would have been a trap a character wide.

append serialises once — those bytes are hashed, written, and read back by anyone who verifies later. compute_hash takes a &RawValue and hashes written.get().as_bytes(). Nothing re-serialises anything.

Record gets a hand-written PartialEq. A raw value has no equality: it is a span of text, and text differing only in whitespace is a different span. That is the right comparison for lines of an append-only file anyway.

Consequences

• Verification no longer requires understanding. A record written by a newer Tacid — a variant this build has never heard of — verifies here, because the hash is over bytes. That is not hypothetical: it is exactly the machine that took an update, failed its health gate and rolled back, whose older keeper now opens a journal the newer one wrote. Before this, that keeper refused the line in read_records, before any hash was checked, and so refused to start. A test writes such a record and asserts the chain verifies, the head matches, and only the reading of that one line fails — by name, carrying what it could not read.
• JournalError::Unreadable is not corruption, and says so. Every reader now decides what an unreadable record means for it, which turns out to be four different answers:
  • Keeper::abandoned, which drives a deletion, propagates: a record it cannot read might be the one saying a generation is bad (ADR-0065).
  • The seal state propagates, because treating an unreadable record as “not a seal” would report that every record of sealing is gone — the signature of tampering — which a record this build merely cannot read is not evidence of.
  • The egress budget propagates, and so the proxy does not start and no byte leaves: a ledger the machine cannot read is a machine that does not know what it has already spent.
  • The first-boot marker and the unattended-changes list drop it. Neither acts; the cost of being wrong is that a machine introduces itself twice, or a person is told about a change twice.
  • The rendered journal shows it: after a rollback, somebody reading their own journal should see that the newer version wrote something here, and what.
• The five frozen digests did not move. They were the test for this change as well as the guard against an accidental one, exactly as ADR-0064 said they would be, and they held.
• The frozen test keeps a narrower job, restated in place: verification no longer depends on Entry, but reading still does, so a shape change still costs the ability to read what is already on disk. It is now a migration about legibility rather than one that stops a machine from booting.
• Reformatting the journal now breaks it. Whitespace inside an entry object used to be normalised away by re-serialisation and is now part of the digest. This is right: the journal is an append-only file of exact lines, not a document to pretty-print, and a tool that rewrote it is a thing the owner should hear about.
• serde_json gains the raw_value feature workspace-wide.

Alternatives rejected

Keep the parsed Entry and hash a canonical form of it. Sorting keys and normalising numbers would make the digest independent of field order, and no more independent of field presence, naming, or the shape of an embedded Principal — the failures ADR-0064 actually found. It also puts a second serialiser in the trust path, which is one more thing that can drift from the first.

Store the whole record line and hash that. Simpler still, and it makes the record’s own field order part of the digest, so reordering seq and id — which carries no meaning at all — would invalidate every journal. The entry is the part with meaning; the frame around it is not.

Make written private and offer only accessors. Cleaner, and it breaks the one thing that needs to construct a Record from parts: the test that forges a shortened chain to prove the counter catches what the chain cannot (ADR-0060). A constructor for the benefit of a forgery is worse than a public field that says what it is.