Roadmap
What comes next and in which order, and the open decisions that need an ADR before the corresponding work starts. Each milestone has a concrete proof.
Status: maintained.
What is done, with its proof, is in ROADMAP_DONE.md.
Milestones
Each milestone with its proof and the steps that reached it is in ROADMAP_DONE.md; here only its state and, while it is open, what is left.
| # | Milestone | State |
|---|---|---|
| M0 | Skeleton builds, tests, sanitizers, static analysis, docs | done |
| M0b | Core tensor engine on the CPU | done |
| M1 | oklm info parses every model of the public corpus (all
quant types, sharded files); oklm quantize output loads in
llama.cpp |
done |
| M2 | Tokenizer bit-exact with Hugging Face on GPT-2, Llama 2 and Llama 3 vocabularies | done |
| M3 | oklm chat with published Q4_K_M / Q8_0 GGUFs (SmolLM2,
Qwen2.5-0.5B, Llama-3.2-1B, TinyLlama) and a safetensors repository,
same greedy output as llama.cpp |
done |
| M4 | Same on OpenCL, faster decode | done |
| M5 | OKLM-Nano trained from scratch generates Shakespeare-like text; GPT-2-124M fine-tune matches llm.c | done |
| M6 | Agent ticks a plan using skills and tools on real files, runs background jobs and agents, permission modes and rules enforced and audited | done |
| M7 | Image, audio and video questions answered locally | done |
| M8 | HTTPS API with dashboard; native GUI | done |
| M9 | v0.1.0 release with OKLM-Nano/Micro weights and tokenizers | done |
| M10 | oklm agent on a par with a full coding-agent CLI |
in progress: every step M0 to M20 done; the gaps left are in gap-analysis.md |
| M11 | Background processes, Monitor, language servers and service integrations for the agent | in progress: P0 to P12 done; left: the evals report and cgroup limits |
| M12 | Git for the agent: context, safe commits, branches, PRs, reviews, worktrees | in progress: G0 to G11 done; left: G11's evals |
| M13 | Slash commands: registry, menu, prompt commands as skills, the full catalog | done |
| M14 | vLLM-style serving: paged KV cache, continuous batching without fixed sequences, prefix caching across requests, preemption | done |
| M15 | ROCm/HIP training backend with LoRA: hipBLASLt products on the
gfx1100, LoRA forward and backward by GEMM composition, then LoRA in the
trainer, adapter files, oklm train --lora,
oklm run --lora |
done |
| M16 | MCP tool-use fine-tuning: a LoRA adapter that drives an MCP tool server through the agent (schema-valid tool calls, multi-step orchestration), on the general default base | done |
| M17 | Retrieval-augmented generation: embeddings, a local index, hybrid search and reranking, retrieval in run / chat / serve / agent with corrective fallback, tool selection by retrieval, then agentic, adaptive, conversational, graph, self-critique and multimodal retrieval | done |
Next
Regrouped 2026-11-08; finished items moved out on 2026-11-11, and the account of finished work inside the open items on 2026-11-15. What is done is in ROADMAP_DONE.md by date, each item with its proof (the 2026-11-07 list by block in 2026-11-06 to 2026-11-08, what was listed on 2026-11-05 in the 2026-11-05 list). Here only what is left: the order and what the work waits for, then housekeeping, small leftovers of finished items, the GPU backend's NVIDIA report, the known speed gaps against llama.cpp (not pursued for now), larger work (the architectures not written, and what the finished model, device and server items leave), what the graphical tools and the chats leave, what is optional, what the user deferred, and last what waits on hardware ("Blocked") or stays out on purpose, with the reason. Item names (A1, E3, H5, ...) are the block letters and numbers of the done history (ROADMAP_DONE.md).
Order of work
The order laid out on 2026-11-13 (the commands and attachments of the chats, speech, the vision projectors, the architectures of L5, the tools on Windows, then the device paths) was followed to its end but for its last part, below.
What this machine carries: 16 threads, 124 GiB of memory, one GPU for the measures (a Radeon RX 7900 XT, 20 GB), 28 to 32 GB of disk free (2026-11-15).
| Resource | Rule |
|---|---|
| GPU | one job at a time holds it (a job paused there is killed);
everything else runs --backend cpu |
| Disk | a model above 2 GB is downloaded one at a time, into the shared
models/, and removed after its check when nothing else
needs it |
| Memory | no limit in practice below 60 GiB a job; one job above that at a time |
| Ports and directories | a job never assumes a port free, never stops a process it did not
start, and runs windows and servers with XDG_RUNTIME_DIR,
XDG_CONFIG_HOME and OKLM_MODELS_DIR of its own
when it writes there |
| Checks | a change runs the tests of what it touched. The whole suite
(make test), the sanitizers (make asan-test,
make tsan-test), cppcheck (make check) and the
wine run close a round of work, once (section 0) |
What is left of that order: the published files and
the measures that wait for disk, one large file at a time (L5, L6, L7,
L9; H3 on Qwen3-Coder-30B-A3B, F1, G1, H6); then G6, I4's other
architectures and G5's glm5-next. No more than two
architectures are written at once: they share
src/model/model.c. After each: every model made runnable is
run on its published file and added to the catalogue (models.md); the documents, the HTML pages and the
presentation are brought up to date; the packages are rebuilt.
What waits for something this machine does not have: V4 for a report from someone with an NVIDIA card; D1 to D3, H8 and T10's Windows list for real Windows and macOS machines; H4 for a model larger than memory; I4's published files for a disk with 80 GB free or more; several of C's leftovers for a person at a microphone and a loudspeaker. A1 and F2 are observations with nothing to change; the speed gaps of section 2 are not pursued; sections 5 to 8 are optional, deferred, blocked or left out.
0. Housekeeping
- The sanitizer round, cppcheck and the wine run last ran on
2026-11-16, before the 1.0.0 tag.
make asan-test(122 test programs),make tsan-testandmake wine-testran whole atc3423e21; what they found was fixed and what failed or changed after that tree was run again ataab876a0, passing: under ThreadSanitizertest_ops,test_agent,test_cli_progress,test_cli_chat,test_server,test_think,test_context_full,test_chat_commandsandtest_compact; under AddressSanitizer and under wine the same withouttest_cli_progress, withtest_threads. Found: a flag read once from the environment that the workers of a product each read and wrote (q8_even_on(), the same value, the five programs ThreadSanitizer failed); two lines the sanitizer builds refused to compile (a layer count sizing an allocation unchecked, a file name copied into a shorter field); under wine, a chunk's 200 microsecond sleep oftest_threadsreturning at once, which left the loop to its caller (the test waits the time out now).make wine-testtakes the default Wine prefix: name one withWINEPREFIXto keep it out of~/.wine. The user runs the round at the end of a day, not per commit. On 2026-11-17, before the 1.0.1 tag, the whole suite, the graphical tools' checks on the CPU and on the device, cppcheck and the documentation's lint ran at78b63947, and the ten test programs changed since 1.0.0 ran again under ThreadSanitizer, AddressSanitizer and wine, passing:test_gui_settings,test_gui_speak,test_agent,test_cli_chat,test_think,test_mem_plan,test_server,test_compact,test_context_fullandtest_chat_commands. Found:test_servercalledstrndup, which the Windows C library does not have, so the tests did not build for Windows (it copies the bytes itself now). On 2026-11-17, before the 1.0.2 tag, the same whole-tree checks and those of the chats on a server ran at78654c07, and the ten test programs changed since 1.0.1 ran again under ThreadSanitizer, AddressSanitizer and wine, passing:test_request_members,test_gui_remote,test_remote_models,test_cli_chat_url,test_sys,test_agent,test_cli_chat,test_chat_commands,test_gui_commandsandtest_gui_settings(test_remote_models,test_cli_chat_urlandtest_cli_chathave no check that runs on Windows). - CI: green at
0775fcc(2026-11-11); the pushes after it to confirm. Its jobpackages(the.deb, the AppImage, the Windows installer for a version tag, attached to the tag's release) was run in adebian:trixiecontainer here on 2026-11-17, not on the server: the first tag after 1.0.1, or a run by hand, is its first run there. That run found two things Debian 13's older compilers refuse and this machine's accept: a model name copied into a shorter field of the chat window (GCC 14), andstatalready named_stat64by MinGW-w64 12's header. The Docker check passes locally (D4). - Disk cleared on 2026-11-15, before the 1.0.0 tag: every published
model of
models/(the samples, templates, tokenizers and datasets stay), the recorded references ofbuild/ref, the 0.1.0 models ofbuild/release-0.1.0, the test folders and the other build trees ofbuild/. A canary set of published files (10.2 GB, one per kind of model,tests/canary_models.txt) was fetched again the same day with its references and is whatmake testcompares; a test that wants another file names it and skips. Every other family is verified when its code changes: its file fetched (tests/public_models*.txt, the catalogue), compared once and removed (testing.md, the entry). A Docker run can leave root-owned files inbuild/.
1. Small leftovers
Each one the "Left" or "Not done" line of a finished item; the detail is in the item, under the link.
- A1. The compact-share rule (0.60 / 0.75 / 0.85 by parameter count) is a choice, no eval measured it: no task here fills a window. A.
- D1 to D3. Job memory and CPU limits, a port's holder and the toast were cross-compiled and unit-tested on Linux, never run on real Windows; D2's macOS branch (libproc) never compiled. D.
- F1.
--n-cpu-moemeasured on a MoE larger than the 20 GB GPU (none inmodels/). F. - F2. Busy machine: OLMoE not re-measured; Granite at two threads under full load with the free-core cap is within the runs' spread, nothing to change. F.
- G1 to G4. No real
glm4moefile (GLM-4.5-Air, about 65 GB, would need--n-cpu-moe);--mtpunchecked (no published GGUF carries the block). G. - H3. Expert cache: the check on Qwen3-Coder-30B-A3B
(18.6 GB, not in
models/); the mirror measured on two NVMe drives (one here). H. - H4. Read-ahead on a model larger than RAM (none here). H.
- H6. The 8-bit draft head (no MTP model fits here) and acceptance against the hit rate on a large MoE. H.
- H8.
test_sys.exebuilt with MinGW, never run under wine; macOS branches (F_NOCACHE,host_statistics64,sysctl) written to the documented API, never built. H.
1b. GPU backend for AMD, NVIDIA and Intel (V)
ADR-0023: Vulkan becomes the main GPU backend; OpenCL stays as the fallback; no HIP or CUDA backend. Targets per device: llama.cpp's Vulkan backend on it (7900 XT decoding: Granite 515, OLMoE 480, Llama-3.2-1B 482, Qwen3-8B 117 t/s, against OKLM's OpenCL 301, 285, 380, 86 to 95; UHD 750: Llama-3.2-1B 15.5 against 4.7). Every op is checked against the CPU kernels, as the OpenCL ones are; AMD (RADV) and Intel (ANV) are tested here.
V1 to V3 are done (ROADMAP_DONE.md, vulkan.md); what is left of the speed targets is in section 2.
- V4. NVIDIA.
tools/gpu/report.sh(build, device tests, the public models against the CPU, a speed table, one report file) for an outside tester; NVIDIA marked untested until a report comes back; fixes and tuning from it. The script exists (testing.md); no NVIDIA report yet.
2. Known speed gaps against llama.cpp
Block 4 ("OKLM against llama.cpp") closed on 2026-11-08 after round 9 (4, benchmarks.md): level or ahead on every CPU prompt row, most CPU decode rows and most GPU prompt rows. What was taken since (the two CPU decode rows left among it) is in ROADMAP_DONE.md (CPU decoding, GPU decoding of the MoE models, Vulkan). The gaps left, each larger than it would return, are not pursued for now; by expected gain:
| What | Why | Expected gain |
|---|---|---|
| GPU decoding of OLMoE and GLM-4.7-Flash: the router's product inside the routing launch (its launch skipped is 371 -> 392 t/s on Granite, an upper bound: the one work-group would do the product), OLMoE's whole-row query and key norms with the rotation, GLM-4.7-Flash's shared-expert gate, up and SwiGLU as one launch, the Q, K and V products of different types as one | OLMoE 0.88 and GLM-4.7-Flash 0.89 of llama.cpp's (Granite 1.09).
What is left is still mostly launches (OLMoE 209 a token, GLM-4.7-Flash
1076; llama.cpp replays a HIP graph; this driver has no
cl_khr_command_buffer), each about 6 microseconds |
about 2 % a launch per layer on OLMoE, less on GLM-4.7-Flash |
| Q4_K prompt tiles: larger tiles, the Q4_K products' share of a prompt (54 to 60 % of its GPU time, attention 13 to 16 %, the Q6_K products 13 to 14 %). Two or eight loader waves a tile and Q6_K tiles with loader waves were tried and did not pay | GPU prompts of Llama-3.2-1B 0.84 and Qwen3-8B Q4_K_M 0.86; the
opt-in F16 matrix units (OKLM_CL_WMMA=1, not the CPU's
bits) run them at 0.98 and 0.99 |
those rows to about 1.0 |
| Vulkan against llama.cpp's Vulkan backend (ADR-0023's targets): Qwen3-8B's decode step, OLMoE's decode launches (the norm into the product, the rotation with the cache writes, the router's product into the routing), prompt attention | bench_llama Q4_K_M on the RX 7900 XT: decoding Qwen3-8B
111.5 against 116.6 (0.96) and OLMoE 465 to 473 against 482 (0.97),
Llama-3.2-1B and Granite ahead; prompts of 512 15 150 against 15 700 and
2 470 against 2 780 (the products alone at llama.cpp's rate; what is
left on Qwen3-8B is prompt attention, about 32 of 207 ms, whose other
tilings measured slower, vulkan.md) |
Qwen3-8B and OLMoE decoding to llama.cpp's. Left: the 8-bit products in a step against the float ones (2 % on Qwen3-8B), an 8-bit gate / up / SwiGLU dispatch for MoE layers, attention's last 3 us a dispatch on heads of 128, the logits' read-back (3 % on the small models, cause not found) |
4. Larger work
Each item holds what is left of it; what was done, with its figures, is under the item's links. A file "not run" for lack of disk waits for its turn there (one large file at a time, removed after its check). The UHD 750 is this machine's second GPU (Intel).
- I4. Architectures colibrì runs that OKLM does not:
mimo2(MiMo-V2.6),inkling,deepseek4,deepseek41,kimi-k3,mistral-large4, each a model item with its own audit (I, the audit,mimo2on a tiny model). None has a published file this machine's disk takes; the table and what is left, in order, are under Scoping. - G5. The GLM-5 series:
glm5-next(GLM-5.3-Flash, linear Kimi Delta Attention, sparse-attention indexer, hyper-connections) andglm-dsa(GLM-5 / 5.1 / 5.2, beyond this machine); about the size of M14 each (G). - G6.
chatglm(ChatGLM3, the first GLM-4-9B-chat) and the GLM-4.5V / 4.6V vision projector (glm4v) (G). - L1. What the vision projectors leave. LLaVA 1.5 and
1.6, Gemma 3, Qwen2-VL and Qwen2.5-VL run and are in the catalogue (LLaVA 1.5, LLaVA 1.6, Gemma 3, Qwen2-VL and
Qwen2.5-VL);
oklm_visionrefuses theclip.projector_typeit does not know (multimodal.md). What their device paths, the server and the Qwen files leave is under L9. Left:- LLaVA: transformers itself not run (the reference is its sources
rewritten in numpy); LLaVA 1.6 on other language models (Vicuna, 34B)
and LLaVA 1.5 13B, not run; files with
clip.vision.feature_layer, refused. - Gemma 3: pan and scan (optional in the model's processor, not in llama.cpp either); a second image in one prompt and an image past the window of 1 024 positions, checked on the tiny model only; the 12B and 27B files (same projector), not run; on a device an image's rows are not compared with a text prompt's rate (2 800 rows a second against the CPU's 214).
- A chat template for a file that carries none (ROADMAP_DONE.md):
Codestral 22B (
mistral-v3) and Foundation-Sec-1.1-8B-Instruct (foundation-sec) run once with their catalogue formats (neither is installed);--chat-templateon an LFM2 file with tools, checked through the agent only (notoklm chat --toolsor a server); a small model whose own template is wrong has no option of its own.
- LLaVA: transformers itself not run (the reference is its sources
rewritten in numpy); LLaVA 1.6 on other language models (Vicuna, 34B)
and LLaVA 1.5 13B, not run; files with
- L3. What Whisper large-v3 and large-v3-turbo leave
(the
models, speed,
timestamps, languages; on a device: L9). Left:
- Speed on the CPU: large-v3's decoder (half its time, bound by memory: quantized matrices would change results); an AVX-512 kernel and tuning on a quiet machine.
- Decoding: temperature fallback, beam search and the previous text as a prompt (a first word at a window's start missed, a sentence skipped once by turbo, an "Ah!" added after silence by large-v3); word-level times; a voice-activity detector; language names beside the codes.
- Reference: no recording compared with whisper.cpp or OpenAI's implementation in timestamp mode (none here: a numpy reference on the tiny checkpoints, and the models' agreement).
- L5. What the models added to the catalogue on 2026-11-13
leave. Each architecture below runs on the CPU and on both
devices, checked against llama.cpp, and is in the catalogue, but for the
last two (audited, no code). Left, by architecture:
qwen35andqwen35moe(Qwen3.5-9B, Qwen3.8-27B, Qwen3.6-35B-A3B and their tunes;qwen35,qwen35moe, kept states, design):- Files: the base Qwen3.6-35B-A3B's Q4_K_M (21 GB) run, with its
catalogue entry (the UD-IQ3_XXS file was run); gate and up joined in one
ffn_gate_up_exps, which llama.cpp accepts and no file looked at has. - Multi-token prediction: the block against llama.cpp's draft pass;
--mtpis not above the plain speed on the CPU (the drafting). - Kept states and the cache: a state kept in a KV file; cuts of a
sequence holding image rows; a journal for linear blocks on a device;
swap-out of a hybrid sequence; a journal with the prompt-start state for
a free
/rewind; the recurrent state is still reserved for the attention blocks too (2.1 MiB per layer and sequence, about 17 MiB a sequence for the 9B); timings on a quiet machine. - The vision projector (
mmproj); the trainer on this layout.
- Files: the base Qwen3.6-35B-A3B's Q4_K_M (21 GB) run, with its
catalogue entry (the UD-IQ3_XXS file was run); gate and up joined in one
- Ternary Bonsai 2 (2B, 4B, 9B; done, design):
the multi-token prediction files (
...-PQ2_0-MTP-Q8_0.gguf: the 4B's ran withoklm run --mtp, not compared with llama.cpp's draft pass; the 9B's not run); thedflashdraft model of the 4B and 9B (audited: six blocks fed with the target's hidden states that draft 15 tokens in one pass, no code); the vision projectors; the 4B and 9BPTQ1_0files (not run); an adapter merged into folded weights (oklm lora-mergerefuses such a file) or tried beside them. gemma4(Gemma 4 12B, 26B-A4B, E2B, E4B; the 12B and the 26B-A4B's form, the small models, tool calls, thinking, inference.md): the 26B-A4B's tool calls throughoklm serve; a shared cache's rows written once to a KV file; a cap on the thinking channel's length; a tool request on the 262 144-token vocabulary takes 8 to 14 s (the grammar's token table suspected, not measured); a comparison past the window of 1024 positions on a published file; the image and audio projectors; the 31B. The rounding of an exact half of a Q8_0 / Q8_1 activation is under "Later".gpt-oss(gpt-oss-20b; done): the reply's channels split in/v1/chat/completions(reasoning_contentandcontent) and inoklm chat, not run on the published file (done on a model made for the test); the CPU speed (3.3 tokens/s: theMXFP4product is portable C); CyberPal 2.0 20B (not run); the 120b.mistral3(Devstral Small 2 24B; done): a comparison beyond 8 192 tokens on the published file (the attention temperature and the YaRN ramp act there; checked on a fixture); the vision projector (Pixtral) of the Mistral Small 3.x files that have one.lfm2(Cyber-Prime 1.1 2.6B; done): the encoder variants (ColBERT, the decision head).lfm2moe(Tini-Cybersec-8B-A1B; done): whatlfm2leaves; its tool calls not run on this file; the 24B-A2B file.starcoder2(StarCoder2-3B; done): the 7B and 15B files; a fill-in-the-middle option foroklm run(the tokens are there, the prompt has to be written by hand).spark2_5(Spark-X2.6 1.7B and 4B; done, inference.md): a comparison beyond 800 tokens (the trained context is 1 048 576); the template's tool calls (the<tool_call>NAME<arg_key>form, not tried); the three reasoning levels as a chat option (the template'senable_thinkingandreasoning_effortare reachable through--template-vars).muse-glimmer(Muse Glimmer 30B, UD-IQ2_XXS; done): the perception encoder (mmproj, 1.4 to 3.8 GB); the template's tool calls (<atem:function_calls>: no parser, no grammar); a file above 2 bits (Q4_K_XL, 15.9 GB); the repository'sdflash-kquant.gguf(not looked at).- The HF conversion of
gemma4,mistral3,lfm2,starcoder2andspark2_5. - The expert cache and the device expert tier on the published files with experts: under L6.
qwen4exp(CYBER-FROST-3.8: 177B parameters, 80 GB at 2 bits): audited from the header and llama.cpp'sqwen4exp.cpp(the audit), no code. It isqwen35moe(the same Gated DeltaNet and gated full-attention blocks, tokenizer and keys) with four things OKLM does not have: hyper-connections (four residual streams of 2560 mixed before and after each attention and each feed-forward by a grouped norm and a rank-320 pair, in place of every layer norm, the final one included), a per-layer n-gram hash embedding on block 1 (a table of 320 001 536 rows of 160, 28.8 GB of the file, read through 16 hashes of the last three tokens), a sigmoid in place of SiLU in the linear blocks' output gate, and a sparse-attention indexer (its tensors are in the file, its ratios are 0: the full-attention blocks are dense); 512 experts, 10 a token, with a gated shared expert. Running it takes those four, a file that fits (none does: 80 to 123 GB against a disk of 15 to 23 GB free), and a smaller model of the family to compare on (Qwen3.8 Flash Next, its base, if a GGUF of it exists).mllama(Llama 3.2 11B Vision): audited (the same entry), no code. The published GGUF is Ollama's format, which the pinned llama.cpp does not read (nomllamathere): 40 blocks, 8 of them (3, 8, ..., 38) cross-attention blocks without self-attention (cross_attn_q/k/v/o_proj, per-head Q/K norms, two scalar gates read through tanh), the image reaching them as 4 tiles x 1601 features of 4096 from a two-stage encoder (32 blocks, then 8 gated global ones, five intermediate layers concatenated) in a 1.9 GBmmproj. Running it takes a reference first (an Ollama build, or the 21 GB checkpoint under transformers); the text-only path is Llama 3.1 8B's 32 other blocks and would be the first step.
- L6. What the device paths of the architectures of 2026-11-13
leave. Each architecture of L5 runs on OpenCL and Vulkan,
checked against the CPU kernels op by op, then on its published file
against the CPU and llama.cpp (ROADMAP_DONE.md, the
entries named L6; inference.md, design). What the
encoders, the sound models and the server leave on a device is under L9.
Left:
- Published files not run on a device (not on disk,
or larger than the device):
qwen35: Qwen3.8-27B and the tunes' Q4_K_M files; aqwen35moefile larger than the device.- Ternary Bonsai 2: the 9B (blocks of 4 096: the op is checked at that
size against the CPU), the 4B and 9B
Q2_0andPTQ1_0files. gpt-oss: CyberPal 2.0; the 120b, which has the layer split alone.muse-glimmer: the Q4_K_XL file (15.9 GB) with its cache.mistral3,starcoder2, DeepSeek-Coder: a file larger than the device.- A context past the window of 512 positions on a device:
spark2_5,muse-glimmer,mistral3(Devstral's attention temperature and YaRN ramp act past 8 192 tokens: checked on the fixture),starcoder2, DeepSeek-Coder. - Streamed CPU layers, on the fixtures only:
lfm2,lfm2moe,spark2_5,muse-glimmer.
- The Intel UHD 750: no published file of these
architectures was run there. What ran: the op tests and the fixtures on
both backends for
gemma4,gpt-oss,spark2_5andmuse-glimmer; on Vulkan, with the op tests alone on OpenCL, forqwen35andqwen35moe; on Vulkan alone for Ternary Bonsai 2. - The expert cache, the device expert tier and
n_cpu_moe:qwen35moe: checked under a hybrid stack on the fixture only.gemma426B-A4B: a context asked with the cache or the tier runs without it, so a file larger than the device hasn_cpu_moealone, whose speed was not measured.gpt-oss: the experts are computed where their layer runs, so none of the three is used (a context asked with them runs without).lfm2moe(Tini-Cybersec-8B-A1B): they work from the llama family's MoE path, not run on this file.
- Multi-token prediction on a device: not run for
qwen35/qwen35moenor for Ternary Bonsai 2. - Ops still on the CPU, or not joined:
qwen35: no journal is written with a linear block on a device, so a cut there is a forward pass from the nearest kept state (fewer than 32 tokens inside what a sequence just generated, 8 for a prompt asked again).- Ternary Bonsai 2: a file folding query, key and value (or gate and
up) unlike, or with blocks above 4 096, is given the CPU with a notice;
the indexed products (
mvid_,gi_,wi_) of the two types have kernels that no test runs (no model has ternary experts). gemma4: a quantized or BF16 cache under the 512-wide heads on Vulkan (those blocks' attention runs on the CPU for these operands: checked as an op, no model run); the joined Q, K, V product is not used for this architecture on a device (two or three products); a model whose blocks share caches takes one device and does not stream its CPU layers.gpt-oss: the expert biases are a gather and an add (two launches a bias), not one indexed add; its routing and gated unit do not fit Vulkan's fused MoE dispatches (about twelve dispatches a layer, 0.8 ms of the step of 6.10); a model whose experts are wider inside than the model runs on the CPU; sinks under heads wider than 256 on Vulkan run on the CPU for these operands (checked as an op); the fixture's down product (16 values wide inside, not a multiple of 32) runs on the CPU for these operands, so the fixture's Vulkan decode steps are not replayed (the published file's are).
- Speed (OpenCL's
decoding on the quantized files, gpt-oss on
Vulkan, Muse
Glimmer's prompts):
- OpenCL's decoding below Vulkan's: Muse Glimmer 29.2 against 36.7 t/s (its products, IQ2_S at 360 GB/s and IQ3_XXS at 450, are 26 of a step's 31 ms of kernel time, and a step is 902 launches); gpt-oss-20b 130 against 164 (its products are at the memory's rate and a block is 25 launches, seven of them a bias added after a product and four the routing, for which no fused kernel was written); Ternary Bonsai 2 4B 163.1 against 180.9.
- Ternary Bonsai 2: OpenCL's prompts are at 0.89 of Vulkan's (the
recurrence is 36 % of the kernel time, serial over the tokens, the
products 38 %); the rotation of folded weights' inputs keeps its stage a
barrier on Vulkan;
PTQ1_0decodes about 2.5 times slower than the two-bit types on a device (its base-3 digits), 3.2 to 3.8 times the CPU. - Muse Glimmer:
iq2_sstays the longest product of a prompt (4.62 ms a launch, 43 % of the kernel time). gpt-osson Vulkan: a segment of 32 invocations for rows of 90 groups (166.8 against 163.6 t/s) changes the order of the sums and was not taken.- An IQ4_NL cache's writes are about 130 us a launch on Vulkan (Llama-3.2-1B decoding at 154.5 t/s against 447 to 456 with a Q4_0 cache: the quantization's sixteen scales, serial in an invocation) and run on the CPU on OpenCL, which has no kernel for them (68.8 t/s, a prompt at 877).
lfm2moe: a tile of the grouped expert products costs the same whatever it holds, so they run at 21 to 23 TFLOPS where a dense prompt product runs at 35 (a tile table packed across experts, or narrower tiles for the experts' remainders, was not tried); the policy between 8 and 16 pairs an expert was not measured.- OpenCL's float matrix units for prompts: a product of the 8-bit kernels' types that they do not take (a row length that is not a multiple of 256 for the K-quants) and float weights stay on the F32 tiles; no device but the RX 7900 XT has the units (the UHD 750 keeps the F32 tiles).
- Not timed: NVFP4 and IQ4_NL products (checked by the op tests); the SmolLM2 IQ files and a Q2_K or Q3_K file.
- Precision, not traced (the
steps where a device parts from the CPU):
qwen35/qwen35moe: the devices' logits are as far from the CPU's as the CPU's from llama.cpp's on the published files (0.11 to 0.16 on average), which the float fixtures do not show (1e-4); not traced block by block.spark2_5: the 1.7B's largest single-logit differences from the CPU (7.4 to 7.8, logits far below the step's best), not traced block by block (the 4B's was).gemma4: OpenCL's rotation with frequency factors is a last place of the angle from the CPU's (1.5e-4 on values of 2 at position 1271, seen in the op test, whose positions stay below 230).
- Observation on a device (observe.md, ROADMAP_DONE.md): with an F16 cache the records of Gemma 4 with experts, Spark, gpt-oss and Muse Glimmer are up to 1e-3 from the CPU's on their fixtures (3.3e-5 with an F32 cache; rounding to halves, not traced further); the lens, the means and a context with parts taken out still read inside the pass; no published file of these stacks but LFM2-350M was compared record by record.
- Published files not run on a device (not on disk,
or larger than the device):
- L7. What the published files that waited for disk
leave (ROADMAP_DONE.md,
Gemma
4 26B-A4B). The files still to run are under L5
(
qwen35moe,muse-glimmer). Left:- Speeds of the 27B and the 30B on a quiet machine.
- The new entries of
test_infer_publicwere measured through a driver calling the same check, not through the committed test binary (the files were removed before). - Still waiting on disk: the measures of H3 on Qwen3-Coder-30B-A3B (18.6 GB), F1, G1 and H6 (section 1).
- L8. Qwen3.5-9B's sensitive steps: the model's own
sensitivity, nothing to fix in the gated delta blocks (ROADMAP_DONE.md).
Left:
- The bound of
test_qwen35_publicon a parted choice for the build for Windows is still the measure (5.0, step 38 of the plain-text continuation; Linux 0.6), no llama.cpp run of seven making that choice: more runs of llama.cpp with its last places changed (another compiler, another machine), or the test following OKLM's own token there. - The same measure on Spark-X2.6-4B's two sensitive steps.
- The per-block comparison tool is not in the tree.
- The bound of
- L9. What the encoders, the sound models and the server leave
on a device. The 24 entries asked for on 2026-11-14 (the
leftovers of L6 and L1) were all taken, each with its entry and figures
in ROADMAP_DONE.md (2026-11-14, 2026-11-15, the entries
named L9; the
last round). The reference pages are multimodal.md, speech.md and server.md. Left:
- Vision encoders on a device (the
encoders' blocks, the
patch embedding and the projector, OpenCL's
first attention kernel):
- Still on the CPU: the image is cut into patches and normalized
there; a projector with quantized matrices or rows that are not a
multiple of 32 values (none on disk); an encoder with quantized matrices
in its blocks is not put on a device at all
(
oklm_vision_use_backend()andoklm_audio_use_backend()answer that they do not take it: no such file, tiny or published, and no test). - A device's matrix products still hand back rows that are not a multiple of 32 values (the encoder pads its own).
- An encoder with BF16 matrices takes the F32 tiles (none on disk; the
tiles checked by the op test alone, the encoder's device path by
test_encoder_typeson the tests' counting device, which computes with the CPU's kernels). - Bidirectional attention: no kernel on OpenCL for BF16 or block-quant keys, heads over 128 or not a multiple of 4, or fewer than 16 rows (one launch a window there, as Vulkan with fewer rows than half a block of its tiles; no published encoder has such shapes), nor for a single sequence of a text encoder over fewer than 128 keys (8.7 to 10.6 us a launch on the first kernel, 10.9 to 20.1 us on the tiles). Gemma 3 4B's and 12B's heads of 256 stay on the kernel of one work-group per head and token, where they measured faster; Gemma 3 27B (heads of 128, the case the tiles take by default) is not on disk: the form is checked by the op tests alone.
- A device without room for an encoder's ends (the patch embedding and the projector, Whisper's convolutions, Ultravox's projector) is tested on the tests' counting device, not on a real small one.
- The retrieval's captioner (ROADMAP_DONE.md): its captions were compared with the CPU encoder's on 6 of the 9 figures and on SmolVLM2-2.2B alone; their texts and the lists of hits were not compared again with the template's prompt; no figure set was run on the UHD 750.
- Still on the CPU: the image is cut into patches and normalized
there; a projector with quantized matrices or rows that are not a
multiple of 32 values (none on disk); an encoder with quantized matrices
in its blocks is not put on a device at all
(
- Speech on a device (speech, the
convolutions, the vocoders,
Ultravox):
- Still on the CPU: the spectrogram of Whisper and of Ultravox (24 ms of jfk.wav's 0.27 s; a transform on a device needs a logarithm no backend has, and was not written); the vocoders' inverse transform (9 to 11 ms for 6.4 s of sound). Whisper's logits are read back each token.
- WavTokenizer's group normalization is five or six launches where an op of its own would be one: not written.
- The single rows of Whisper's cross-attention stay on OpenCL's first attention kernel, by the figures (keys and values in F16 would halve a launch's read and round the text's inputs: not tried).
- The device's text was compared with the CPU's on the tests' samples only (jfk.wav and the fixtures' signal).
- The first column of OpenCL's vocoder table (the transform on one thread) was not run.
- OpenCL against Vulkan on the encoders (ROADMAP_DONE.md):
OpenCL's products are F32 where Vulkan's matrix units take F16: 0.14 of
the 0.28 s of kernel time of a Whisper window (jfk.wav with
large-v3-turbo 0.35 s against 0.27 s) and 0.23 of Gemma 3's 0.82 s (its
4 096 rows 0.74 s against 0.69 s, float attention 0.40 s of it).
OKLM_CL_WMMA=1gives them the units and still rounds an encoder's activations. - Precision of the encoders (ROADMAP_DONE.md):
- Ultravox's embeddings on OpenCL are 2.4e-4 from the CPU's (the single running sum of the tiled products: rounding, kept) and 7.2e-5 on Vulkan with the convolutions and the projector there, where the blocks alone gave 5.5e-5.
- A device's embeddings part from the CPU's as the rows grow (3e-4 at most measured).
- The greedy answers do not come closer to the CPU embeddings' with the wider products (47 of 48 for both Qwen files with the language model on Vulkan: the language model's own prompt products still round to F16, by design); Qwen2-VL 2B's answer from the OpenCL encoder's embeddings agrees with the CPU embeddings' on 46 of 48 choices (a near tie, 0.18).
- A device with cooperative matrices other than the RX 7900 XT was not run.
oklm servesharing a device (the encoders on a server's device, a pass cut between blocks, the room, a spread context):- Gaps left in another request's stream: the language model's step
over an image's rows is not cut (LLaVA 1.5 7B 0.19 s for its 576 rows,
0.06 to 0.09 s with
--prefill-chunk 64, not a default; Qwen2.5-VL 7B 0.16 to 0.17 s for 391 rows; Gemma 3 0.08 to 0.09 s); a block of a large picture is 0.19 to 0.22 s (Qwen2-VL at 3 024 embeddings) and is not cut inside; on OpenCL one gap of 0.30 s was seen once with LLaVA 1.6 and once with the large picture, not traced. - The recognizer's and the synthesizer's threads open no stream of
passes, and the quarter of a second still stands for them; the
scheduler's stream was tested on fake devices
(
test_pass_stream), not timed on a model slower than four steps a second; a spoken sentence's gap does not tell the synthesizer's tokens from its vocoder's call. - The room counted: the reserve for a backend's own buffers is one
figure a backend (243 to 244 MiB of Vulkan's stay unused with an encoder
whose passes are short: LLaVA 1.5, Ultravox, Whisper; a reserve by the
rows of the largest pass was not written), and OpenCL's query sees 5 MiB
of the 38 a first pass takes; a vocoder stays counted at its longest
call (2 048 frames), which a sentence does not reach (1 161 MiB counted
for OuteTTS 0.2 and its vocoder, 1 070 to 1 078 taken; Soprano 348 and
270 to 274); a generator used as a reranker is counted by the same
function and was not run;
GET /api/contextand a dry run still read the device's free memory with the cache in place, and leave a draft model's room out; the refusals at start are tested on the tests' counting device, not on a real small one; a smaller device was not tried, the published models were counted on the RX 7900 XT only. - The retrieval's own encoder computes on the CPU in a server
(
oklm rag indexputs it on a device, outside any server). - A context spread over several devices:
test_multi_devicepasses without the change too on this machine (no test provokes another pass coming between a spread pass's ops on its first devices); a served model spread over the two GPUs was run by hand only; OpenCL instances of two different devices were not run (the first device opened twice). - The published files went through the server on both devices for the stream gaps; their answers were not compared.
- Gaps left in another request's stream: the language model's step
over an image's rows is not cut (LLaVA 1.5 7B 0.19 s for its 576 rows,
0.06 to 0.09 s with
- Pictures, videos and saved sequences
(
oklm describe, the chat routes; KV files, videos, a video in the server, several pictures, the layers split):- After
--kv-load:--vision-idis refused (the file keeps no count of pictures); the context is not sized for a new picture (-c); a next turn's sound was not run on a published audio model. - The server's prefix directory with an image was not run with a served model of three-part positions (no tiny file of one has a tokenizer).
- A video on the published Qwen files: only an answer was read, no reference existing; Qwen2-VL 2B's answer to a vaguer question about the video, with half its layers streamed, parted from the CPU's at up to 5 of 48 tokens (near ties); a split over two devices was not run.
- A video in the chats: in the graphical chat the file dialog, a drop
and the path's field were not driven with a video, nor the press of
Transcribe (the option that calls it was), and the
window's own recognizer was not run on a video; the page sends no
fpsorframes; the page's video case was not run in WebKit; the table of decoded parts lost at a restart and the fall back to decoding when the embeddings were dropped are not driven by a test; the frames as still images were run on SmolVLM-256M, not on LLaVA 1.5 or Gemma 3. - The "Picture N:" labels: the Transformers processor's default was
read in the template and not in its code (not installed here);
/v1/messageswrites none; the labelled answers were compared with no reference.
- After
- Against llama.cpp and the models' own code (the
template's prompt, the
RMS norms' epsilon, the
CPU's encoder at large pictures):
- Gemma 3 4B with the template's prompt: OKLM and llama.cpp's whole path agree on 44 of 48 tokens (47 against llama.cpp given OKLM's embeddings); the two programs' encoders are 0.3 to 0.7 % apart, and where they part was not traced.
- Qwen2.5-VL against the float64 pass of its own code: the 32B and the 72B (their files are not on the disk), whole answers on the BF16 file and of the prompts with several pictures.
- On the CPU Qwen2-VL's encoder at 4 015 embeddings stays a fifth behind llama.cpp (its attention over 16 060 rows), measured on a machine with other work.
- Qwen2.5-VL 3B, 32B and 72B: one quantization a size, the 72B at 1.75 bits only; no video or several-picture prompt on these sizes.
- The Intel UHD 750 (ROADMAP_DONE.md):
- Run there: SmolVLM-256M, SmolVLM-500M, Qwen2-VL 2B and Whisper base, each slower than on the CPU. Not run: the larger vision files, the large-picture, several-picture and video cases, a larger recognizer, an Ultravox file, a published vocoder, a served published model (the fixtures' test ran, both backends), the server's gaps.
- Why Intel's OpenCL driver runs the row kernels and the tiled ones alike at four to five times Vulkan's time on the same device (about 17 GFLOP a second; matrix products are 92.5 % of Qwen2-VL 2B's prompt) was not found; no kernel was changed.
- Vision encoders on a device (the
encoders' blocks, the
patch embedding and the projector, OpenCL's
first attention kernel):
4b. Graphical tools (T)
One oklm-gui binary with an entry point per tool
(oklm-gui --tool NAME), each with its entry in the system's
menu (gui.md). T0 to T9 are finished (under
"Closed"); T10 (the registry and --listen on Windows, the
nine tools under Wine, servers kept across logins, activations in the
inspector) is done but for what follows (the
registry and the listener, the
tools under Wine, servers
kept across logins, the
Windows leftovers, activations,
the
inspector on devices, the lens and ablation).
T10. Left on Windows itself (run under Wine 11 only): the access lists of the registry's folders, a console's Ctrl-Break, a real pid used again, the system's own save dialog and a drop from Explorer (run through Wine's), the certificate store, a sound card for Record, the browser opened by Documentation and Dashboard, the installer's shortcuts, a real login running the
Runvalue, UTF-8 text in a real console.T10. Left under Wine or there: file names above 255 UTF-8 bytes and paths beyond
MAX_PATH; the window's font has no CJK glyphs; the server manager's Keep across logins box on Windows (the command line was driven); dragged text on Windows; the retrieval reranker, graph and captions; the conversion's Stop in the script; the inspector's views on Windows (compiled, not run).T10. Kept servers: no restart on failure on Windows; macOS never built (launchd is written); a reboot or a real logout and login not gone through on any system; Re-keep with form in the server manager not driven under Wine, its two pictures older than it.
T10. Elsewhere: nilorea's
n_spawn_*andn_open_with_defaultstill call the narrow Windows API (the tools no longer go through them there); a text field of the training window drew hyphens as blanks in one screenshot (not looked into).T10. Activation capture (observe.md): the recording's cost on OpenCL (the reads are batched, but a queued copy costs that driver's host side what a blocking read did: recording still takes a tenth to a third of the speed there; fewer commands a token would be the gain); the attention rows' host arithmetic (a key row converted once a head that reads it); the lens's logits and the means' rows still read inside the pass; the staging read and the logits read as one wait; only the first device's blocks staged; the inspector's file run on a device; Effects with the mean in place of zero; encoders and multi-token-prediction contexts.
T11. The options of
oklm servein the server manager and the chat window (done on 2026-11-15: the offload row, the Advanced window shared by the two tools, the chat window's model settings with the restart it asks for, a server's sampling defaults taken when the window attaches; then, the same day, what that left:-nand--seedas a server's defaults, the index's reranker in the chat window, the memory line on a device, the settings kept for each model, the buttons pressed without a person, eight pictures taken by a script, the Windows build under Wine; then its leftovers: a model loaded in place with the settings kept for it, the options only a device has checked from the form, the two scripts that clicked rewritten on the tools' presses, every picture of the chat window taken by the script, the memory line judging as the server sizes its cache, with a control for the cache's size; then, on 2026-11-16, three of its points: the memory line counting what shares the memory with the model, every group of the Wine check on the tools' presses, a model loaded in place with nothing kept for it; then, the same day, five more: the embedding and the reranking models counted on the device their backends name, the models that compute on the CPU counted with what they work with, a speech model the chat window will load, a model's options of the command line automatic again at a switch, the form, the server manager and the chat window checked to say the same of one server; a chat window started with nothing named, which loads no model and asks what to talk to; then the memory line's four gaps, a model larger than its device, a switch that never ends with nothing loaded; and what was decided of its leftovers: a model chosen there judged and loaded in the window, the saved servers, the agents and a file in its list, the options without a control and the MoE disk options in the chat window, the Advanced window 740 pixels wide, the window that asks under Wine); then a model its device holds with less than one context of cache, which has fewer layers there in place of a refusal. Left:- A start with nothing named. Use this
model, in the Models window of a window that serves nothing,
still starts the window again on the model: only the window that asks
loads in place. A window linked to a server or an agent that is given a
model there starts again too. The catalogue's model case of
tools/check_gui_start.shneeds SmolLM2-135M undermodels/and is skipped without it; the row of the system's file dialog is pressed under Wine only (tools/check_gui_wine.sh, groupstart), and the projector's dialog nowhere (its row is given a path by the test aid). - What the memory line does not size: an encoder is
counted on the model's device even when the model keeps no layer there,
as the server reserves it; a transcription or a speech model on a device
is counted there without what it keeps in host memory (the recognizer's
arenas, the synthesizer's context left on the CPU); the copies a
projector's encoder makes of a few of its tensors (the patch kernel, the
projector's matrix) are not in its working memory; one card reached
through two backends (OpenCL and Vulkan) is judged as two devices, each
by its own figures; a model beside the model on a second device was run
with an embedding model (the Intel UHD 750 beside the RX 7900 XT), not
with a sound model, and no real second device was short (the counting
device's is, in
test_side_device); what an embedding model on that integrated device keeps in host memory was 17 595 904 bytes where the line counted 14 433 792. The vocoder's call is counted at its longest (a whole prompt's codes), which a reply spoken piece by piece does not reach. - A server whose model does not go whole on its device, or
leaves it less than one context: the layers of such a model go
to one device (
--devices 2spreads them by the devices' sizes, not by what the cache leaves). A server that runs keeps its layers where they are: a larger context asked of it (POST /api/context, the Context slider of a selected server) gets the cache that is left beside them, not the layers a start at that context would place, and its line is red there while the form's is not. The layers are chosen against what the device has free at the start: while another program holds most of it, none may leave it a context, and the model then runs on the CPU (the line and the server's log say0 of 36 layers). Host memory short of the layers and the cache that stay there is refused by the plan and the line and only warned of byoklm serve, which starts; it was checked with the plan's figures, not on a machine short of memory. In a chat window a model loaded in place is no longer refused for the device's room alone (not loadedis checked at a start with--kv-mem, not at a switch on a device). - The presses on Windows itself.
tools/check_gui_wine.shis Wine's run; its sound group needsultravox-1b, which is not of the canary set. The system's file dialog is answered from the keyboard there, which a desktop that gives the dialog no focus would not take. - The MoE disk tab: the memory line counts the expert
cache and what is pinned in it, in both tools;
--expert-packand the expert options of the Other field are not in the count. In the chat window the cache was run on the tiny mixture of experts on the CPU, not on Granite 3.1 nor on a device (the line and a server's own figure were compared on Granite 3.1, from the server manager's form). - In the chat window: the embedding and reranking
models are refused while the conversation holds a long document, and
compute on the CPU whatever
--embed-backendand--rerank-backendsay (greyed there); nothing is kept for a server's model, nor for the models beside the model (the projector, speech, retrieval). The Other field takes the options the window's own server can be given (gui.md), not the names, the keys, the adapters nor the formats of a server; what each of them changes in the server was not measured (the model restarts with them and answers). - Options without a control (the list in gui.md): they stay in the text field.
- A start with nothing named. Use this
model, in the Models window of a window that serves nothing,
still starts the window again on the model: only the window that asks
loads in place. A window linked to a server or an agent that is given a
model there starts again too. The catalogue's model case of
4c. The chats: commands, attachments, speech (C)
Every item is done, C11 the last, each with its proof in ROADMAP_DONE.md (the entries named "The chats", the latest being a full context, then the window's and the dashboard's leftovers, the small leftovers and the model switch, the words while speaking, the documents kept, and C2's media in the terminal chat). C5 (files in the dashboard) and C6 (models that are not chat models) leave nothing. What is left of the others:
C1. One command table.
oklm chat's line mode on Windows reads whole lines: the line editor is POSIX's, and what it needs there (the console's modes, waiting on the console handle) cannot be checked without a Windows console.
C2. The commands.
- Media in the terminal chat: not run on a device nor under wine;
/modelis refused while the chat holds media;/decidereads the conversation's text, not its pictures;--resumeencodes each media file again (the files keep the rows, not the embeddings); the context is not grown for a picture asoklm describe's is (/context N); no--vision-idlabels;/export jsonholds the text only; LLaVA 1.5 and 1.6 and Qwen2.5-VL not run in the chat. - The terminal's model selection: its download checked on a device.
- The window: Agree and download pressed on a model whose licence asks (those are above 500 MB); a failed download.
- The model switch of a running server: a load that fails after the
model served was freed, and the restore that follows, were run on the
tests' counting device, not on a card (the plan refuses what does not
fit before anything is freed, and the driver gave memory beyond the
card's when the card was said larger than it is); a server with
--observeor--prefix-diralways frees first; the server manager's rows after a switch not looked at in its window; its lock (one switch at a time, taken before the requests' gate) and the documents' lock held while the reranker's is taken have not been through ThreadSanitizer.
- Media in the terminal chat: not run on a device nor under wine;
C3. Files in the chat window.
- A paste on Windows (a picture's, and copied files'
CF_HDROP) is written and cross-compiled, not run; a file of another machine's share is not attached. Copied files are read on X11.
- A paste on Windows (a picture's, and copied files'
C4. Documents.
- No corrective step (no judge of the passages, no second search).
- A question about the whole is answered from a sample of the document (no summary of every part in turn), and is told by English words.
- The server keeps its conversations' indexes in memory only.
- An attached agent gets the passages of the message the document came with and is not searched again for the next ones; one asked before its model is loaded tells no context and is sent the whole text.
- The reranker's effect on answers was not measured.
C7. Speech to text.
- Needs a person or another system: a voice in front of the microphone (the test machine's input gave silence; recordings were played as the microphone); a microphone chosen on ALSA alone and on Windows; the recorder run on Windows; Record and Talk in WebKit (its permission dialog cannot be answered by the check).
- The partial texts are the recognizer run again, not a streaming decoder: with a large model on the CPU they come too late to be of use (large-v3-turbo: a word waits up to half a minute). They were run through the window's own server, not with the window's own recognizer nor a remote server.
C8. Text to speech.
- Needs a person or another system: heard from a loudspeaker; the pause of a real device's stream (run on the file sink); Safari itself (WebKitGTK was run).
- A second real speaker file with the window's own synthesizer.
- Soprano: its decoder untuned (two fifths of its time), its package's
hallucination detector, the author's PyTorch package and the real
inflect/unidecodeas references, Soprano 1.0, one tokenizer count unexplained (29 tokens where the published table says 30, as llama.cpp); its F16 file is not on this machine, so its two rows of speech.md's tables are an earlier session's. - OuteTTS 0.3: against llama.cpp itself, its first word lost in one run, its punctuation tokens, its sound at temperature 0 (close to silence).
- Another language (tried with OuteTTS 0.3 and the English voice: German 13 of 19 words heard, French 5 of 17).
mp3,opusandaac(refused with the reason); FLAC without LPC or the header's MD5, and not used by the window.
C9. A spoken conversation.
- Needs a person: echo cancellation heard with a loudspeaker and a microphone, the adaptive level in a real noisy room, the mode with a person at a microphone.
- An attached agent talked with.
- Cloning a voice (no encoder of WavTokenizer and no aligner here).
C10. A full context.
- A model that writes its reasoning in a channel: continued through a
server on Gemma 4 E2B alone among the published files; gpt-oss and Muse
Glimmer stay verified on a tiny model each (they are not in the canary
set, and are fetched only when their own code changes). A cut inside the
first header of a turn was run on the three tiny models, through the
server and in
oklm chat, not on a published file (Gemma 4 is not kept on the disk). The chat window and the in-process agent were not run on such a model: the agent's rule for a header cut is written and not run (a tiny model's markers are control tokens, which end an agent's turn). A harmony reply cut inside the header of a call's message (commentary to=..., before the call's marker) was not run. - A reply cut inside a tool call: not for Llama 3's bare JSON nor
GLM-4's name line, and not with
oklm agent --no-tool-grammar, by decision: without a marker that opens the call or a grammar that reads it, a half call cannot be told from text. - The dashboard's chat: a message the server refused for its length
and answered in a context made larger was not run (a reply the context
cut was); in WebKitGTK what
/exportwrites is read from the page, no download followed; Safari itself was not run. - The agent, room for a turn (done):
nothing is judged before the model counted a prompt, so the first
request of a conversation still goes to a window that may be too small
for it (one request, refused or cut short, then the message that says
so). A server other than
oklm servethat refuses a prompt for its length stays an error of that server's. Not run: a sub-agent in a window without room; the larger context on a device; the full-screen UI's show of the message. oklm chat's full-screen UI: a piece asked for later with/continuefollows its own line of the transcript and starts a block (as reasoning when the reply was cut inside its reasoning; an open code fence is not carried over)./retryand/rewindof a continued reply were not typed there (they take the conversation's message, which the check holds to be one reply).- In the chat window on a server at an address, a compaction counts by
the server's tokenizer where the server has
count(done); the texts of the summary's requests are counted one request each, and what the conversation takes after a compaction is still told from the estimate until the next reply's count. - A continued reply read aloud: heard by Whisper, not by a person; the
voice of a server at an address was not run across a continuation (the
window's own, and its own server's, were); a reply stopped with
Stop has no Continue, so nothing follows it for the
voice. A sentence is cut at
Mr.and other abbreviations (a short pause). A conversation with pictures was compacted with SmolVLM-256M on the CPU, its two pictures in one message; a picture in each of several messages, a sound and a video were not.
- A model that writes its reasoning in a channel: continued through a
server on Gemma 4 E2B alone among the published files; gpt-oss and Muse
Glimmer stay verified on a tiny model each (they are not in the canary
set, and are fetched only when their own code changes). A cut inside the
first header of a turn was run on the three tiny models, through the
server and in
C11. A server that is not OKLM's (decided and done 2026-11-17). The agent talked to any OpenAI-compatible server (
oklm agent --url); the chats now do. Proved on llama.cpp's server (the pinned build of the test reference), nothing leaving the machine.oklm chat --url URL [--api-key KEY] [--remote-model NAME]: the terminal chat on such a server, with the agent's options and its HTTP backend. Done 2026-11-17, with the list of point 3 in the terminal chat (ROADMAP_DONE.md, inference.md). Left:- what needs the model in this process is said and not done on a
server: tool calls (
--tools,/tool),/decideand/rag(anoklm servehas/v1/decideand its retrieval stages, which the chat does not ask), a picture, a sound or a video (a server that takesimage_urlis not sent one);--repeat-penaltyis not sent; - the conversation's size is an estimate on every server: an
oklm servecould count it (thecountmember of a chat request), and another server's window is a setting, not read from it (llama.cpp's/propshas it); - a cut reply goes back closed, with a request to go on, only after a server answered 400 or 422 to the open one: run on the test's own server; llama.cpp's server (0.8.0-dev, build 8642541) goes on with the open message, with and without reasoning, so it never took that way. A server that reads the open message as a closed turn answers with a new one, which the chat cannot tell from a continuation;
- a reply cut inside its reasoning is asked again, not gone on with: a server sends the reasoning apart and takes none back;
- automatic compaction on a server that is not OKLM's was run on the
test's own server; on llama.cpp's server
/compactwas; - not run: an
httpsserver, a hosted endpoint, the Windows build (the test is POSIX only, astest_cli_chatis).
- what needs the model in this process is said and not done on a
server: tool calls (
- The chat window on such a server: done 2026-11-17
(the
window on a server that is not OKLM's). The checks reach no hosted
endpoint, on purpose: a key and the text of a conversation would leave
the machine. The refusals such a server may answer (a member it does not
know, a reply left open, no count of the tokens) are run on the check's
stand-in, and a hosted server is the owner's to try. A sound's
input_audiopart is checked on the stand-in, which decodes it: no published audio model was served by a server that is not OKLM's, since the canary set has none that llama.cpp's server loads with an audio projector. Such a server's own transcription and speech routes are not used, by design: the window transcribes and speaks itself. - The models of
GET /v1/models: a list to choose from in both, in place of a name typed. In the chat window: done 2026-11-17 (the same entry). Nothing is left there. - Endpoints kept with their key: done 2026-11-17 (the
same entry). Left: on Windows the file is given an access list of its
owner alone, which was run under Wine: the calls succeed and the window
saves, reads and forgets a server there, but Wine keeps no access list
(what it reads back is made from the Unix mode), so that the list is
protected and holds the owner's entry alone is Windows' own to show. Not
in it: a client of Anthropic's
/v1/messages(prompt caching, reasoning kept across tool calls), worth its work only for a regular use of Claude through OKLM.
Scoping of the open items
The scoping notes of the open items above; their history is in ROADMAP_DONE.md.
G5. GLM-5 series, two architectures, both far heavier than the above (sizes from their configs; experts are most of them):
glm5-next, GLM-5.3-Flash (about 305B, 13B active: 45 layers, 288 experts of 2048, 8 a token, 1M context, a vision tower): hybrid Kimi Delta Attention linear layers (a short causal conv, a per-channel gated delta rule: a variant of the Gated DeltaNet kernels OKLM has) with multi-head latent attention without rotary on the full layers, a DeepSeek sparse-attention indexer with k-pool compression (top 2048), manifold hyper-connections (several residual streams mixed by Sinkhorn-normalized matrices) and DeepSeek MoE. Each is new to OKLM; together about the size of M14. About 175 GB at Q4: over 128 GB RAM and 20 GB VRAM together; at 2 to 3 bits it fits only with--n-cpu-moe(F1).glm-dsa, GLM-5 / 5.1 / 5.2 (about 745B, 78 layers, 256 experts):deepseek2plus the sparse-attention indexer (DeepSeek V3.2's), full on some layers, and an MTP block. Beyond this machine at any precision; the indexer is the shared piece, worth it only withglm5-next.
G6. Later, optional:
chatglm(ChatGLM3, the first GLM-4-9B-chat), the GLM-4.5V / 4.6V vision projector (glm4v).I4. Architectures colibrì runs that OKLM does not, each a model item like G with its own audit. They were audited on 2026-11-15 from the first megabytes of their published GGUF files (metadata and tensor table, the rest left on the server), the pinned llama.cpp (
build/llama.cpp, v0.8.0) and the names of colibrì's sources. Sizes are those the repositories listed that day; the machine has 124 GB of memory, a 20 GB GPU and 28 to 32 GB of free disk, so "here" is decided by the disk.Architecture Published files (smallest set) What OKLM does not know Reference Work Runs here mimo2(MiMo-V2.6-Flash, Pro; 48 blocks of 4096, 256 experts, 8 a token)ggml-org/MiMo-V2.6-Flash-MOPD-GGUF105.9 GB (Q2_K_S),-Flash-RL-GGUF126.4 GB (Q2_K), 167.4 GB (MXFP4)heads of 192 for queries and keys and of 128 for values ( attention.value_length),attention.head_count_kvandattention.sliding_window_patternas lists whose first block is a full one (4 key heads on a full block, 8 on a window block),rope.freq_base_swa,attention.value_scale;attn_qkvof those widths,attn_sinkson the window blocks; three multi-token prediction blocks in a file of their own (mtp-*.gguf, 1.26 GB)llama.cpp mimo2.cpp(398 lines); colibrìmimo.cthe smallest: the llama block with pieces OKLM has (sinks, window, fused projection, sigmoid router with exp_probs_b), new are the value width and the two lists. Written on the CPU and checked on a tiny modelno: 105.9 GB (its tensor table loads and binds). Its distillation into Qwen3.5-9B is qwen35and runs (mimo-v2.6-distill-qwen-9b)inkling(Inkling, Inkling-Small; 42 blocks of 4096, 256 experts, 6 a token, 2 shared),inkling2(Inkling-2-Small)unsloth/Inkling-Small-GGUF74.8 GB (UD-IQ1_S); Inkling 270.2 GB; Inkling-2-Small 171.7 GBrelative positions in the attention ( attn_r,attn_rel_proj,d_rel,rel_extent), short convolutions on the inputs of the attention, the keys, the values and the feed-forward (shortconv_*),ffn_gscale, logits scaled by the logarithm of the context (log_scaling_*), theinklingpre-tokenizer;inkling2adds an attention gate, clipped queries (attn_q_clip,attn_r_clip), a projected sink, residual scores (attn_res.*) and a scale per expertnone in the pinned llama.cpp (the name is unknown to it); colibrì inkling.c; the model's own codelarge: a new attention; no second program to compare with unless llama.cpp is moved no: 74.8 GB deepseek4(DeepSeek-V4-Flash; 43 blocks of 4096, 256 experts, 6 a token)unsloth/DeepSeek-V4-Flash-0731-GGUF82.5 GB (UD-IQ1_S) to 155.1 GB (UD-Q4_K_XL)four residual streams mixed by Sinkhorn-normalized matrices ( hyper_connection.*,hc_*), keys and values compressed 4 and 128 to 1 (attention.compress_ratios,attn_compressor_*), a sparse-attention indexer (top 512), a window of 128, low-rank queries and a grouped low-rank output (attn_q_a/b,attn_output_a/b), routing by token id on the first three blocks (ffn_gate_tid2eid), clamped SwiGLU, sinks, gating function 4llama.cpp deepseek4.cpp(1 510 lines) and its own cache (llama-kv-cache-dsv4.cpp, 2 436 lines); colibrìdeepseek_v4.cthe largest; hyper-connections and the indexer are G5's too no: 82.5 GB deepseek41(DeepSeek-V4.1-Flash; 40 blocks of 5120, 384 experts)unsloth/DeepSeek-V4.1-Flash-1017-GGUF233.0 GB (UD-IQ1_S), 110.6 GB of it one n-gram tabledeepseek4's, plus an embedding of hashed 4-grams added on two blocks (engram.*,ple.*: 48 hash heads of some 16 000 000 rows each), blocks reading other blocks' keys (attention.kv_source_layers), indexer candidates by blocks of 8llama.cpp deepseek41.cpp(933 lines overdeepseek4); colibrìdeepseek_v41.cafter deepseek4no kimi-k3(Kimi K3; 93 blocks of 7168, 896 experts, 16 a token)unsloth/Kimi-K3-GGUF466.4 GB (UD-Q1_0)Kimi Delta Attention blocks ( ssm_*,kda.*: three of four blocks), latent attention on the fourth, experts in a latent of half the width (expert_latent_length,ffn_routed_*), residual scores (attn_res.*), thesituactivation, a tensor type numbered 66llama.cpp kimi-k3.cpp(616 lines) anddelta-net-base.cpp; colibrìkimi_k3.clarge; the delta attention is G5's glm5-nexttoono. Small checkpoints exist as safetensors only ( tiny-random-kimi-k3, a 0.4B)mistral-large4(Mistral-Large-4-1T-A52B; 72 blocks of 6144, 128 experts, 4 a token)unsloth/Mistral-Large-4-1T-A52B-GGUF233.0 GB (UD-IQ1_S);ggml-org/...297.7 GB (IQ2_XXS)latent attention as deepseek2with an indexer on some blocks (attention.indexer.types, top 2 048), a scalable softmax (attn_ssmax), rotation on three blocks out of four (attention.rope_pattern), blocks reading block 35's keys (attention.kv_source_map), a logit soft-cap, thetekken-num3pre-tokenizernone: the pinned llama.cpp has mistral4,deepseek2's graph under another name, and does not knowmistral-large4; of colibrì onlymistral_vision.hcarries the namedeepseek2plus the indexer (G5's) and three small piecesno Left, in this order:
mimo2, written and checked on a tiny model (inference.md): a published file run against llama.cpp (105.9 GB at least: MiMo-V2.6-Flash-MOPD Q2_K_S; with the experts streamed from disk the resident part is 8.85 GiB, so it is the disk that is missing, not the memory), then its catalogue entry; OpenCL and Vulkan (the values' padding and the attention output's narrowing as ops, or an attention op with a value width; until then a context asked on a device runs on the CPU, and that fallback was not run with a device); the tokenizer against llama.cpp on the published vocabulary (pre-tokenizerqwen2, 152 576 tokens, NFC); the model's tool-call form (<tool_call><function=NAME><parameter=KEY>); the three multi-token prediction blocks (mtp-*.gguf); files with separateattn_q,attn_k,attn_v; adapters, the inspector andoklm quantizeon this architecture (not tried); the HF conversion; the cache's value cells at the keys' size (192 places a head where 128 would do).inkling(a reference first: the model's code in float64, or a newer llama.cpp).deepseek4thendeepseek41(with G5: they share the hyper-connections and the indexer).kimi-k3(with G5'sglm5-next).mistral-large4.
A published file of each waits for a disk with 80 GB free or more.
5. Optional
- MCP fine-tune, round 3. The pipeline works; the training did not carry to the held-out tools and a second round on the model's own successes added nothing. Worth it only with examples written for the failure seen (an unseen tool called directly, not through Bash), measures of 50 tasks or more in place of 15 and 9, and data not only from the model itself (mcp-training-design.md).
- Longer runs of the released models. 6000 more steps bought 0.045 (Small) and 0.149 (Micro) of held-out loss and no visible change; only more pretraining data would.
6. Deferred: the security fine-tune
Deferred on 2026-11-07, after two runs, so the rest of the roadmap goes first.
- Security fine-tune (was 13; "Default models" step 4). Two LoRA runs were made on 2026-11-07 (security-training-design.md): the second learned its corrections and cut the held-out false positives, but missed held-out vulnerabilities the base found (most of its corrections were clean twins). Left: balanced corrections, the adapter at its lowest held-out loss, more snippets per class, then the held-out measure again.
7. Blocked
| Item | Blocked by | Unblocked when |
|---|---|---|
The hybrid Gated DeltaNet layers' ssm_* parity against
real Qwen3-Next weights |
the only release is 80B-A3B, beyond this 20 GB GPU | a smaller Qwen3-Next, or a larger GPU |
The Windows cmd.exe shell and graceful stop
verified |
wine cannot run a quoted .bat or propagate
call's environment |
a real Windows machine |
| The job tests on Windows | the same | the same |
| The Android checklist on a physical device | no device (the emulator checklist passes) | a device attached |
8. Left out on purpose
| Item | Why it stays out |
|---|---|
| LS | Glob and Bash list files; another tool costs prompt tokens for nothing new. |
oklm agent mcp serve |
Exposing the agent's tools to other clients is another product; nothing here uses it. |
/keybindings, /terminal-setup |
The bindings are fixed and documented; a file and its parser add surface no one asked for. |
/voice |
Speech to text exists, but recording needs an audio capture dependency and a push-to-talk interface. |
/subtask |
Background sub-agents and /fork cover it. |
/fast, /advisor |
A second model; smallModel covers the cheap path. |
| MCP sampling, elicitation | Sampling lets a server drive the local model, a security surface with no use here; elicitation needs an interface flow. |
| Managed lockdown of hooks, MCP servers and plugins | An administrator's feature; the managed permission switches exist, the rest has no administrator to serve. |
| A cloud identity reminder, credential redaction, a tunnel's URL parsed | Cloud CLIs are read-only by default and tunnels always ask; these would be polish, not safety. |
| Network exceptions per integration | Listening ports and daemon sockets in the sandbox reopen what it closes; asking each time is the safer default. |
| Image content blocks in the agent | Needs the multimodal agent; goes with pasted images, under "Later". |
| More than the spec's seven tools loaded | On purpose: ten are deferred, the rest earn their tokens. |
| Git: an issue workflow, pushes in CI, more for large repositories, compact git output | Reads and asked writes are enough; workflows belong to skills, not the core. |
| The iOS simulator | macOS only. |
| Pictures above 4 096 embeddings for Qwen2-VL and Qwen2.5-VL (the models' published limit is 16 384) | Decided 2026-11-14: no reference encodes it, and it is sixteen times the attention of 4 096 (ROADMAP_DONE.md). |
| The Qwen video processors' own frame sampling (two frames a second, at least four), and Qwen2.5-VL's seconds a pair truncated as transformers 4.57.1 does | Decided 2026-11-14: --fps 1 --frames 8 stays the
default, and the published 4.49.0 form is kept (ROADMAP_DONE.md). |
| Partial sums in OpenCL's tiled products | Decided 2026-11-14: they would bring Ultravox's embeddings to 1.1e-5 from the CPU's for 7 % more time in every float prompt product (ROADMAP_DONE.md). |
| Vulkan's encoders back on products rounded to F16 by default | Decided 2026-11-14: the wide products stay the default;
OKLM_VK_WIDE=0 gives the earlier rounding and speed (ROADMAP_DONE.md). |
| LLaVA's encoder as the pinned llama.cpp reads it (the class token after the patches, one block fewer) | OKLM computes the encoder as the model defines it (multimodal.md). |
Closed
Decided not to do, or done differently, with the reason:
- NEON kernels: OKLM is not used on ARM yet; a cross toolchain exists
in
/opt/build/armbin/if that changes (see "Later"). - PyTorch golden tensors: gradients and activations are checked by finite differences, in-test double-precision references, the numpy float64 reference and llama.cpp / llm.c built as test tools (adr/0015-test-references.md).
- ASan / TSan inside GPU drivers: the drivers leak and start threads
the sanitizers cannot follow; OKLM's OpenCL code is checked for memory
errors by
make asan-opencl. - Jinja template inheritance,
includeandimport: they need a template loader, which chat templates never have. - Splitting tensors across OpenCL devices (row split): each device's context is separate, so without peer copies every matrix product would send its input to each device and gather their output rows through the host. Estimated on Llama-3.2-1B (no second device to measure): decode would add about 7 products x 16 layers x 2 transfers of ~30 us, 6.7 ms a token against 4 ms for the whole token on one device; a 512-token prompt about 70 ms of transfers against 100 ms of compute. Layers are split instead (one hand-over per device per batch), with weight streaming for the layers left on the CPU.
- CPU speed parity with llama.cpp: closed at 1.1 to 1.2 times for some
weight types, ahead for others (Llama-3.2-1B, 512-token prompts, 8
threads, 2026-11-01,
llama-benchon the same files: Q3_K_M 642 / 704 t/s, Q5_K_M 679 / 773, IQ4_XS 693 / 833; Q4_K_M 597 against 513 to 690 and Qwen2.5-0.5B Q4_K_M 1275 / 853 ahead; 3.5 to 5 times slower before the kernel work). Every CPU layer's quantized weights are repacked for AVX-512 VNNI and AVX2 kernels, bit-identical to the scalar ones; the attention, softmax and SwiGLU run vectorized (benchmarks.md). What is left is the kernels' raw speed: they are bound by their instructions (one thread's speed does not depend on K, eight threads give 7.3 times one), and the remaining ideas measured slower on this i7-11700K: an activation layout with fewer broadcasts, 512-bit registers, Q4_K in byte form, eight activation rows for the K-quants but Q5_K. Another CPU may tell otherwise (see "Done", item 11). CPU decoding is level with llama.cpp's or ahead from four threads (benchmarks.md). - On-device kernel auto-tuning (kernel-autotune-plan.md): done for
what can be tuned without changing a result, the CPU kernels' work split
(
oklm tune, a bitwise logit check, a persistent cache keyed by machine, model and kernel sources, applied by contexts; tuning/). The OpenCL thresholds each pick another kernel or reduction order, so no candidate would pass the check; they stay manual knobs. Tuning runs on request rather than on first load (a library should not spend a minute unasked). - OpenCL KV cache writes fused into their producers: removing one of the two cache writes of each layer (an upper bound, measured by skipping it) gains 1 % (Llama-3.2-1B) to 3 % (Qwen2.5-0.5B) of generation speed, not worth an op with two destinations; Q and K share one rotation launch, and rotated or quantized caches put kernels in between anyway.
- Hand-written fast paths for pre-tokenization: profiling shows it does not matter. Encoding runs at 0.5 to 2.5 million tokens/s on one thread (benchmarks.md), a 2000-token prompt in 1 to 4 ms, against seconds of prompt processing (under 0.1 %).
Finished with nothing left, taken out of "Next" on 2026-11-15:
- RAG (M17): every step (K0 to K9) and every follow-up, each with its measure (ROADMAP_DONE.md); what the evaluation has not measured is under rag.md.
- T0 to T9, the graphical tools (gui.md): the groundwork, machine-readable progress, retrieval and model layout (T1 to T3: training.md, rag.md, convert.md), the server manager, the retrieval manager, the training window, the model inspector, conversion and quantization, the sessions browser.
- L4, DeepSeek-Coder 1.3B and 6.7B (the 1.3B, the 6.7B, against a float64 pass).
- L9's entries that left nothing: Qwen2.5-VL's windowed blocks in one launch, gpt-oss-20b's perplexity between the backends.
Default models
Decided on 2026-11-02 (agent-design.md, decision 5), changed on 2026-11-04: the 27B first chosen (Qwen3.8-27B) is too large for this machine (20 GB GPU) to train or to run with a long context, so the target is an 8B of the same family.
| Use | Model | Source |
|---|---|---|
General use and coding (oklm agent,
oklm chat) |
Qwen3-8B: one model serves both, so the agent rarely switches models | the official BF16 safetensors, Qwen/Qwen3-8B |
| Base for security training | Qwen3-8B abliterated (its refusal direction removed, its capabilities kept) | made by OKLM from the official weights above (see 3) |
| Quick runs and measurements | Qwen3-4B (on disk, Q4_K_M) | the same family: the same blocks, tokenizer and template |
| The small model (spec §2.5) | Qwen3-0.6B | the same tokenizer and template |
Qwen3-8B is a dense qwen3 model (36 layers, 4096
channels, its own output head, vocabulary 151,936): the blocks the
trainer and the inference engine already run, so no new layer type is
needed (hybrid linear-attention layers are a task of their own, "Done",
item 1, its real-weight check "Blocked"). About 16.4 GB in BF16, 8.7 GB
in Q8_0 and 5 GB in Q4_K_M. Unit tests keep their tiny synthetic models
(seconds).
The weights. Training starts from the official BF16
safetensors, never from a community GGUF: a community file is a degraded
copy whose making OKLM does not control. The frozen base of a LoRA run
on the GPU is a file OKLM quantizes itself from those weights (Q8_0, or
Q4_K_M with an importance matrix), trained with
--quantized-base, which keeps it in that type on the device
(training.md). Community GGUFs may serve to
compare inference with llama.cpp and nothing else.
The abliterated model. OKLM makes its own from the official weights (step 3). Published abliterated versions of Qwen3-8B, if any, are to be surveyed in step 1 as references to compare against (their refusal rate and KL divergence the figures to meet or beat), never as training bases. The Heretic GGUFs of the 27B first listed here (refusals 6.0 % on Zou et al.'s prompts, first-token KL divergence 0.0467) are no longer the reference.
The steps this implied (loading and checking the model, making it the default, the abliteration, the small model) are done, in ROADMAP_DONE.md; the security training (step 4) is deferred, under "Next".
Later
Deferred on purpose: neither scheduled nor refused (what is refused is in "Next", "Left out on purpose").
NEON kernels. OKLM is not used on ARM yet (a cross toolchain is in
/opt/build/armbin/if that changes); not scheduled.Training multimodal models (encoders and projectors), and images pasted into the prompt (clipboard or a dragged path, which needs the multimodal prompt in the agent). Inference on multimodal models is done (M7); training them, and pasting images in, are not scheduled.
Rounding of 8-bit activations. OKLM rounds an exact half of a Q8_0 / Q8_1 activation away from zero (ggml's portable code); llama.cpp on x86 rounds it to even with 127 / max. Left as it is on 2026-11-13: it shows on the first block of files with Q8_0 embeddings only (4.6e-4 on Gemma 4 26B-A4B), and changing it would move the bits of every such model, the pinned references and the devices' 8-bit kernels.
OKLM_Q8_ROUND_EVEN=1rounds as x86 does, for comparisons (inference.md).
Open decisions
None: ADR-0007 (video containers) and ADR-0018 (the HTTP API) are accepted.