Sandboxed local engines behind traits; bring-your-own-key providers; no default

Context

The mind is portable. Local models must be good enough that the owner does not immediately export an API key, and cloud models must be reachable without the agent ever holding the key.

The three plans

• Grok: offline by construction; llama.cpp, whisper.cpp, Piper; profiles tiny/core/lab; a small model fine-tuned on Tacid’s tools beats a big generic one; cloud off by default with a NET LLM badge.
• Fable: local for STT, classification and small edits; frontier cloud for planning and code; Anthropic, OpenAI and xAI via a first-run picker, no silent default; CUDA/ROCm/Vulkan.
• Codex: a runtime abstraction with llama.cpp-class inference as the floor; native provider adapters, not a lowest common denominator; bring your own key; privacy profiles; cost visibility.

Decision

Local engines (llama-server and whisper-server; a Kokoro TTS server once somebody has read how it listens — see the note below; mistral.rs later) run as sandboxed subprocesses behind Rust Backend, Stt, Tts and Embed traits: read-only model files, seccomp, no network, embedded chat templates ignored (they parse hostile files; review). Backends: CUDA on NVIDIA; Vulkan by default on AMD with ROCm opt-in (the ROCm closure is large and often broken; review); Vulkan on Intel; CPU everywhere. Constrained decoding (JSON Schema → GBNF) so small models call tools reliably. A signed registry (models/registry.toml) with license, source, hashes, sizes, RAM/VRAM, backends and tasks; profiles tiny/core/lab sized from detected hardware; weights never in the netinstall ISO.

The note, added after reading the two servers at the versions this machine pins. “Sandboxed subprocess on a Unix socket” is a real constraint and not every engine meets it. llama.cpp v0.3.0 does: tools/server/server-http.cpp treats a --host ending in .sock as an AF_UNIX path and binds that string. whisper.cpp v1.9.2 does not: examples/server/server.cpp calls bind_to_port(hostname, port) and has no AF_UNIX path at all — and since tacid-infer runs with RestrictAddressFamilies = [ "AF_UNIX" ] and PrivateNetwork = true, an engine that can only bind a port cannot start under it whatever it is told. Nothing has read how a Kokoro server listens.

So the supervisor plans only engines whose listening it has read, and says why for the rest, rather than starting a process that exits every time and reporting a crash loop. And for one that cannot bind a socket it holds the socket itself, in front of a port the engine binds inside the unit’s own network namespace — which is ADR-0004’s design for it, written now (ADR-0086). What is still refused is an engine nobody has read.

Cloud: bring your own key, no default provider, keys in the keeper and injected by the egress proxy. Two adapters first, the Anthropic Messages API and the OpenAI-compatible family (OpenAI, xAI, Google’s compatibility endpoint, OpenRouter, llama-server); native adapters follow when a provider’s capabilities diverge enough to matter; the trait stays rich. Privacy class derives from provenance and path policy, never from a classifier. Badge and ledger on every request; cost shown and never claimed authoritative. Offline is a state, not a failure.

Consequences

• Five native adapters on day one would be five APIs for one maintainer; two cover almost all.
• Resource governance: eviction on memory pressure; VRAM yielded to owner workloads.
• Model choice is a registry entry, never an architectural constant.

Alternatives rejected

• Ollama as the architecture: a product dependency; it can be a registry source.
• A Tacid-hosted inference gateway: a business, an account, a liability.
• Linking inference in-process: hostile parsers inside the agent’s address space.