56. A local model is held to a grammar compiled from the tools it was offered

Status: accepted. Refines ADR-0016, which said constrained decoding would happen and did not say how.

Context

ADR-0016 says the local mind gets “constrained decoding (JSON Schema → GBNF) so small models call tools reliably”. That sentence has been in the tree since the founding and nothing implemented it. What stood in its place, until it was removed, was --grammar-file /dev/null on the llama-server command line: wrong in kind, because a server-wide grammar would constrain the sentence a person is being answered with as surely as it constrains a tool call, and wrong in fact, because an empty file is not a grammar and the server would most likely have refused to start. Nothing would have noticed either, because no test on this machine starts an engine.

Removing it made the gap honest and left it open. Reaching for it now turned up that the gap is larger than a missing flag.

llama-server without --jinja does not read the tools field at all. The catalogue this machine assembles — every tool the session may see, minus every tool the keeper denies, plus the two the protocol always offers — is serialised into the request, sent, and discarded. The model is never told what it may call. What comes back is whatever the weights felt like emitting, and the adapter looks for it in tool_calls, which nothing has filled in.

So the local path was not “tool calling that is unreliable”. It was tool calling that could not happen, on a machine whose whole claim is that it works offline.

--jinja is the other way out: it makes llama.cpp use the weights’ own chat template to format tools and parse calls back. Tacid does not honour that template (ADR-0016, and --chat-template chatml is passed to override it), because it is the model author’s opinion about how this machine should assemble a prompt, and because a template that parses tool calls is a parser this machine did not write standing between a model and a privileged API.

Decision

For a local endpoint, and only a local endpoint:

The catalogue goes in the prompt. The tools are written into the system turn, each with its name, its sentence, and its JSON Schema as JSON. A schema is easier for a small model to read than a paragraph about a schema, and prose would be a second description that can drift from the first.
The sampler is held to a grammar compiled from the same list. tacid_model::grammar compiles the catalogue to GBNF whose root admits exactly one of: a call, {"tool": "<name>", "arguments": {…}}, with the name a literal alternative per tool and the arguments compiled from that tool’s schema; or an answer, {"say": "…"}.
It is sent per request, in the request body. Not --grammar-file. What constrains a tool call must not constrain the sentence a person is being answered with, and the two arrive in the same field — so the root rule admits either, and the grammar is rebuilt for each turn from the tools that turn is allowed.
The reply is read out of the content. There is no tool_calls to read, so the adapter parses the object itself, after trying tool_calls and only when the endpoint is local. A remote reply that happens to be a JSON object is text, as it always was.

One list feeds the catalogue and the grammar. The two ways they can disagree are both silent: a tool in the grammar and not the catalogue is one the model never learns about, and a tool in the catalogue and not the grammar is one the sampler forbids the model to name.

What the compiler understands, and what it does with the rest

Objects with required and optional properties, arrays, strings, integers, numbers, booleans, enumerations, const, $ref, and anyOf/oneOf.

The last three were not there at first, and leaving them out was not a small gap. The proposal tool’s schema is generated from Vec<ChangeItem> — the whole closed set of things the agent may propose — which is a union behind a $ref. Without those two, the most important tool on the machine compiled to “any JSON at all”, and anything narrowed into its schema was thrown away on the way to the sampler. const matters for the same reason one level down: a variant’s tag is {"const": "service_enable", "type": "string"}, and reading the type first made it “any string”, so a model could write a tag nothing can read.

A $ref becomes a rule of its own, with the name reserved before the target is compiled. That is what turns a type containing itself — NixValue contains NixValue — into a rule referring to itself, rather than a compiler that does not terminate.

Everything else emits value, which admits any JSON: a pointer that resolves to nothing, and allOf, which is an intersection of grammars and not a thing GBNF does. The safe direction, and deliberately so: a grammar that guessed narrowly at a combinator it does not implement would make a legal call unemittable, and the model would look stuck rather than wrong. The authority on what arguments mean is the typed protocol they are parsed into and the keeper’s verdict on the transaction that results — never the grammar. The grammar’s job is to make the envelope certain, not to replace the type system behind it.

The catalogue is narrowed into the schema, and so into the grammar

A service name is the one field where “any string” is provably wrong: a name that is not on this machine’s allow-list cannot be built, so offering it means the change is composed, rendered, staged and put to Nix before anybody finds out. The machine carries that allow-list at /etc/tacid-system/services.json, and it reaches a model three ways: as a fact in the system turn, as an enum on the name of every service operation in the proposal schema, and — through this compiler — as the alternatives the sampler cannot leave. The first is a suggestion, the second is what a remote provider validates against, and the third a local engine cannot get around.

The enum is injected by keying on the discriminant, not on where the generator put the definition: a walk that knew the shape of schemars’ output would break silently the next time the derive changed its mind, leaving a model free to name a service that does not exist. An empty allow-list narrows nothing, because a machine that has said nothing has not said “nothing”.

A tool name is untrusted input

Third-party MCP servers are untrusted integrations (ADR-0018), and their tool names reach this compiler. A name is put into the grammar as a literal by way of serde_json::to_string and then escaped for GBNF — two escapings, in that order, for two readers. A name written to close the literal it lands in, close the rule, and open one of its own is a name with quotes in it. This is the Nix-literal test in another language, and it is a test.

Consequences

A small model on this machine can call tools, and cannot emit something the machine cannot read. The failure mode moves from “emitted something that is not JSON” to “asked for the wrong thing”, which is the failure the keeper is built to catch.

Rule names are positional first (call-0-facts-get), so two tools cannot collide however they are spelled. The JSON primitives are defined unconditionally and then pruned if the root does not reach them: defining them lazily is how a grammar ships with a dangling reference the first time a schema grows a field, and leaving them in is bytes on every request. Only the primitives are pruned — a rule compiled from the catalogue and not wired into the root is a bug in the compiler, and pruning that one too would turn the check that finds it into a check that tidies it away.

What is proven. That a local model cannot emit a service name this machine does not implement — the whole chain, from the file the system carries to the alternatives in the grammar, which is the test that found the union and the reference missing. That the compiler produces a grammar whose every reference is defined and whose every rule is reachable from the root — the two ways a generated grammar is wrong, and the strongest claim available without an engine. That a hostile name cannot introduce a rule, and round-trips back to itself. That required fields are required and optional ones are optional in any combination. That the catalogue and the grammar are compiled from one list. That a constrained reply becomes the same typed completion the remote path produces from a different field, that a local engine which ignored the grammar is still heard, and that a remote reply which happens to be JSON is not read as a call.

What is not proven. That a real llama.cpp accepts what this emits. Nothing in this container starts an engine. That test stays written down as unbuilt rather than implied by a green suite, and tacid-infer’s supervisor says so where the flag used to be.

What is not done. Nothing constrains a remote model, which needs nothing: that family guarantees the tool-call shape itself and has no grammar field. A local server that does honour tools — vLLM, say — is sent both, and told the same thing twice rather than two different things.