OKLM documentation
A dependency-free LLM runtime and training stack in C17: load, convert, quantize, run, train, fine-tune, serve, and drive models as an agent. Version 1.0.3.
Start here
OverviewA self-contained LLMOps framework: a dependency-free LLM runtime and training stack in C17.
Documentation mapOKLM is a self-contained LLMOps framework: a dependency-free LLM runtime and training stack in C17. One source tree loads, converts, quantizes, runs, (more)
Guides
How it worksA guided tour of OKLM for someone who knows a terminal but is new to the inside of language models. For each topic: the idea, how OKLM does it, and (more)
The OKLM agent: user guideHow to use oklm agent, the coding agent of OKLM, day to day. Each section points to the page with the details: harness.md for how it works, (more)
Extending the OKLM agentEvery way to add to oklm agent: skills and prompt commands, sub-agent types, hooks, MCP servers, plugins, declared services, and the library API for (more)
Project
Research: what OKLM builds onThis page summarizes the formats, reference implementations and model architectures OKLM re-implements, with the details that matter when writing the (more)
ArchitectureHow OKLM is layered, which module owns what, and the rules every module follows. The core layer exists today; the layers above it are designed here (more)
Building OKLMHow to build the library, the tools and the tests, which options exist, and every environment variable OKLM reads at run time.
TestingHow OKLM is tested: one binary per module, double-precision references, documented tolerances, and every run repeated under AddressSanitizer + UBSan (more)
BenchmarksMeasured performance of OKLM's kernels, with the machine, the command and the date of each run, so regressions and improvements can be tracked. (more)
OKLM 1.0.1 release notesWhat changed between 1.0.0 (2026-11-16) and 1.0.1: the leftovers of the graphical tools' options and of the full context, decided point by point, and (more)
OKLM 1.0.2 release notesWhat changed between 1.0.1 (2026-11-17) and 1.0.2: the chats on a server that is not OKLM's, the packages built and published by CI, and two defects (more)
OKLM 1.0.3 release notesWhat changed between 1.0.2 (2026-11-17) and 1.0.3: the defects of the sources that the jobs of CI find and the checks of 1.0.2 did not run, and (more)
OKLM 1.0.0 release notesWhat OKLM 1.0.0 holds, what it promises to keep across the 1.x versions, the platforms it was tested on and its known limits.
OKLM 0.1.0 release notesThe first models trained from scratch by OKLM, with its own trainer, on one machine, and the files to run them with OKLM or llama.cpp.
RoadmapWhat comes next and in which order, and the open decisions that need an ADR before the corresponding work starts. Each milestone has a concrete proof.
Roadmap: doneWhat the roadmap planned and is finished, each item with its result and its links (the proof), moved here from the roadmap on 2026-11-11 so that the (more)
Plan: On-Device Kernel Auto-Tuning Pass for the LLM InterpreterGoal: add a Magnitude-style optimization pass that, on first load of a model, tries alternative configurations of each hot kernel on the actual (more)
Open-Source LLM Training DatasetsReference list of free, openly licensed datasets for pretraining, fine-tuning, and aligning a local LLM. All datasets load via the Hugging Face (more)
CompatibilityThe live matrix of open-weight model families, tensor types and tokenizer pre-tokenizers OKLM can load and run. "Verified" means checked in CI (more)
Modules
Core: types, tensors, backends and CPU opsThe core layer is the tensor engine everything else runs on: element types, half-precision conversion, block quantization, tensors and views, the (more)
Converting and inspecting modelsHow oklm info, oklm convert and oklm quantize work, what they produce, and how their output is checked against llama.cpp. API: include/oklm/io/hf.h, (more)
TokenizerHow OKLM turns text into token ids and back, which tokenizer families and file formats it reads and writes, how it is checked against Hugging Face, (more)
OpenCL backendHow OKLM runs inference on a GPU through OpenCL: how the runtime is found, how a device is chosen, where tensors live, the kernels, the fallback to (more)
Vulkan backendHow OKLM runs on a GPU through Vulkan (ADR-0023): how the API is loaded, how devices are chosen, how tensors are addressed, how shaders are built and (more)
TrainingTraining a transformer from scratch, fine-tuning a loaded model and resuming from checkpoints: the trainer's API, its kernels, checkpoint files, the (more)
InferenceHow OKLM runs a decoder-only transformer from a GGUF file: loading and binding the weights, the context and its KV cache, the Llama-family and GPT-2 (more)
Mixture-of-experts modelsHow OKLM runs a mixture-of-experts (MoE) model, whose feed-forward layers route each token to a few of many experts: the grouped expert products, the (more)
Activations of a running modelWhat OKLM can record of what happens inside a model while it evaluates tokens: how much each block writes to the residual stream, where the attention (more)
Models: the catalogue and its downloadsOKLM ships no model file: not in the repository, not in the .deb, the AppImage or the Windows installer. oklm models lists the models it offers to (more)
Memory planWhat a model would take before anything loads: oklm plan for one model or every model of a directory, and the warning run, chat and serve print when (more)
Command referenceEvery command of oklm and every tool of oklm-gui on one page: its synopsis, what it does, what each option means, and the page that describes it in (more)
OKLM agent commandsGenerated by oklm agent commands --markdown from the command registry; do not edit by hand.
Git with the OKLM agentWhat oklm agent knows and does in a git repository, and what it asks before doing. The design is in git-design.md, the details in agent-tools.md.
Processes, services and integrationsHow oklm agent runs things that outlive a command: background jobs, monitors, wakeups, language and MCP servers, declared services, containers, (more)
HarnessThe agent harness behind oklm agent: the loop that runs a task, the tools the model calls, what decides whether a call runs, the settings and memory (more)
Agent loopHow oklm agent runs a task: the loop of model turns and tool calls, the system prompt, the backends and their stream events, headless and library (more)
Agent toolsThe tools oklm agent's model calls: the built-in tools, skills, deferred tools found with ToolSearch, sub-agents and the git operations. Part of the (more)
Permissions, sandbox and auditWhat decides whether one of oklm agent's tool calls runs: the permission rules and modes, the calls that always ask, the audit log of every decision, (more)
Settings and memory filesHow oklm agent is configured: the settings files and their order, every setting, the usage counts, the debug log and the telemetry export, and the (more)
Slash commands and the terminal UIWhat the user of oklm agent types and sees: the slash commands and the full-screen terminal UI. Part of the agent harness (harness.md); every command (more)
Context, compaction and sessionsHow oklm agent keeps a conversation within its context (accounting, old tool results cleared, summaries) and on disk (transcripts, resuming, (more)
Hooks, plugins and MCP serversHow oklm agent is extended from outside: hooks that run on its events, plugins that bundle commands, skills, agents, hooks and servers, and MCP (more)
ServerThe HTTP API: a model behind OpenAI-compatible endpoints (and an Anthropic-compatible /v1/messages beside them), a dashboard, access rules, and the (more)
ROCm/HIP Training Backend — Implementation Plan---
MCP Tool-Use Fine-Tuning PlanMCP client support in the agent (the agent specification's §16, so the exact tool-call wire format is known) and on LoRA training (rocm_spec.md R7).
RAG Spec: Retrieval-Augmented Generation for OKLM and its Agentimplemented, each with its proof in rag.md (what was built differently is in rag-design.md's "As built" and "Implemented" sections); K9: the (more)
Retrieval (RAG)Retrieval-augmented generation in OKLM: text embeddings, documents cut into chunks and kept in one index file, searched by vectors, by keywords or (more)
RAG designHow rag_spec.md maps onto OKLM: the modules each step extends, the files it adds, the stage contracts, the public API, the command line and server (more)
Plan: vLLM-Style Serving Mode for a Custom LLM FrameworkA phased plan for adding high-throughput, multi-request serving to an existing LLM inference framework. Each phase leaves you with something that (more)
Serving designHow serving_spec.md maps onto OKLM: the module each phase extends, or where new code goes, the decisions the spec leaves to its S0, and the baseline (more)
Native window and API clientoklm-gui, the native graphical tools (Allegro 5 and nilorea's n_gui) in one program: the launcher, the chat window, the model manager, the server (more)
MultimodalImages, sounds and speech given to a language model: decoding, the vision and audio encoders whose embeddings a model reads in place of tokens, (more)
Speech synthesisA text spoken into a sound: a language model writes audio codes from the text, a vocoder turns the codes into a waveform. This page is the audit of (more)
Formats
Block quantization formatsByte-level layout and decoding rule of every block-quantized type OKLM reads or writes. These are ggml's formats (ggml-common.h for the layouts, (more)
GGUFByte-level layout of GGUF files as OKLM reads and writes them, the validation rules the reader applies, and how sharded models are handled. API: (more)
safetensorsLayout of safetensors files, the validation OKLM applies, and how sharded Hugging Face repositories are opened. API: include/oklm/io/safetensors.h.
Decisions
ADR-0001: Language and dependenciesRecords the language, the compiler settings and the complete list of dependencies OKLM may use, and why everything that touches the model is written (more)
ADR-0002: Weight formatsRecords which weight formats OKLM reads and writes, and that it follows the public specifications exactly instead of defining its own.
ADR-0003: JSON on the model pathRecords how OKLM parses the JSON it meets while loading models (safetensors headers, config.json, tokenizer.json, index files).
ADR-0004: Regular expressions for pre-tokenizationRecords how OKLM evaluates the regular expressions that byte-level BPE tokenizers use to split text before merging.
ADR-0005: KV cache precisionRecords which types the KV cache can hold and how a quantized cache is kept accurate.
ADR-0006: Chat templatesRecords how OKLM turns a conversation into the prompt text a chat model expects.
ADR-0007: Video in containers other than OggRecords how OKLM reads video it cannot decode itself, and what it runs to do so.
ADR-0008: Training checkpointsRecords how a training run is saved and resumed: a model file that every tool loads as it is, and a sidecar next to it with the rest of the state.
ADR-0009: Tool-call wire format and constrained decoding grammarRecords how OKLM finds tool calls in a model's answer, how it represents them, and how generation is constrained so the calls are well formed.
ADR-0010: Permission rulesRecords how the harness decides whether a tool call runs: the modes, the rule syntax, where rules come from and which one wins, and what is audited.
ADR-0011: Sandbox per platformRecords how the commands the harness runs are confined on each platform, and what happens where confinement is not available.
ADR-0012: Job persistence and orphansRecords what the harness keeps about background jobs and agents, what happens to them when the harness exits or restarts, and how no process is left (more)
ADR-0013: K-quant write pathRecords how OKLM writes K-quants and picks per-tensor types, so its quantized models are interchangeable with llama.cpp's.
ADR-0014: nilorea-library used as a source treeRecords how OKLM consumes nilorea-library: by compiling the source files of the modules it needs, not by linking a prebuilt library or vendoring a (more)
ADR-0015: Test references without PyTorchRecords how numerical tests obtain their expected values while PyTorch is not available on the development machine.
ADR-0016: CPU worker teamRecords how the CPU backend runs parallel loops: on a team of threads kept alive between operations, added to nilorea-library as N_PARALLEL_TEAM.
ADR-0017: OpenCL backend designRecords how the OpenCL backend fits into OKLM's tensor engine.
ADR-0018: The HTTPS APIRecords what oklm serve exposes, who may call it, and how it shares one model between concurrent requests.
ADR-0019: The paged KV cacheRecords how the KV cache is split into blocks shared between sequences: the block size, the layout per layer, how quantized rows fit, the prefix (more)
ADR-0020: LoRA adaptersRecords how LoRA adapters are named, trained, saved and applied.
ADR-0021: adapters served beside the weightsRecords how several LoRA adapters are served in the same batches (serving S8.5). It amends ADR-0020's "Applying" decision, which only merged one (more)
ADR-0022: The RAG index fileRecords how a retrieval index is stored: the container, what identifies the embedding model, the chunks, the vectors, the BM25 postings, how the file (more)
ADR-0023: Vulkan as the main GPU backendRecords how OKLM runs on AMD, NVIDIA and Intel GPUs: which API carries the fast path, what stays of the OpenCL backend, how a decode step avoids its (more)
Designs and specifications
Agent designHow the coding agent of agent_skill_chat_spec.md maps onto OKLM: how the model is run today, and for each section of the spec the existing module it (more)
Agent specificationsThe four specifications of the coding agent, which one owns each part they share, and the order to build them in. Each part is specified once, in its (more)
Agent Spec: A Claude Code–Equivalent on My LLM InterpreterFill in before starting:
Commands designHow commands_spec.md maps onto OKLM. What exists, with evidence, is the "Commands" section of gap-analysis.md: two REPLs with a handful of string (more)
Commands Spec: Slash Commands Like Claude CodeFill in before starting:
Gap analysis: the coding-agent specification against oklm agentEvery section of agent_skill_chat_spec.md, then of process_spec.md, git_spec.md and commands_spec.md, against what the harness already does, of (more)
Git designHow git_spec.md maps onto OKLM. What exists, with evidence, is the "Git" section of gap-analysis.md: the harness has no git code yet, so this is (more)
Git Spec: Working with Git in My Local LLM AgentFill in before starting:
Hybrid linear-attention layers (Gated DeltaNet)How OKLM gains the linear-attention layers that newer bases (Qwen3-Next, and the roadmap's "Next" item 18 step 5) mix with full attention, so the (more)
MCP tool-use fine-tuning design (T0 audit)How mcp_training_spec.md maps onto OKLM: what the stack already gives for teaching the default model to drive an MCP tool server, so T1–T3 reuse (more)
Process designHow process_spec.md maps onto OKLM: the module each section extends, or where new code goes. What exists, with evidence, is the "Processes & (more)
Process & Services Spec: Background Processes, Monitor, LSP and Service IntegrationsFill in before starting:
ROCm designThe audit (R0) of rocm_spec.md: what OKLM has for a ROCm/HIP training path and LoRA, what each later phase reuses, extends or creates, and the five (more)
Security training design (defensive)How the "Security training" step of the roadmap's "Default models" is scoped and built. Decided defensive (2026-11-05): the fine-tune teaches the (more)
Tuning
Kernel tuning: codebase mapThe survey that kernel-autotune-plan.md asks for before any tuning code: how OKLM is built, which kernels run on the forward path, where they are (more)
Kernel tuning: tolerancesHow the tuner decides that a candidate gives "identical output" (kernel-autotune-plan.md, harness and oracle).
Kernel tuning: parametersWhat the tuner may change, and what it may not (kernel-autotune-plan.md).
Kernel tuning: resultsIntel Core i7-11700K (8 cores, AVX-512 VNNI), 8 threads, 2026-10-31, oklm tune -m <file> --backend cpu (512-token prompt, 60 s budget):