OKLM documentation
OKLM is a self-contained LLMOps framework: a dependency-free LLM runtime and training stack in C17. One source tree loads, converts, quantizes, runs, trains, fine-tunes, serves and drives models as an agent, with no external library touching the model (the optional ones serve the edges: TLS, the GUI window, video decoding).
Table of contents of the OKLM documentation. Every page starts with a one-paragraph purpose and a status line.
Status: maintained.
Guides
Start here: pages written to be read through, which point to the references below for the details.
- how-it-works.md: a guided tour for newcomers to LLM internals: model files, quantization, an end-to-end quantization run, memory, backends, generation, the server, the agent, training, multimodal, refusal analysis and the developer tooling, how and why.
- user-guide.md: using
oklm agentday to day. - extending.md: adding to the agent: skills, commands, sub-agent types, hooks, MCP servers, plugins, services, the library API.
Project
- research.md: the state of the art OKLM builds on: formats, reference implementations, model architectures.
- architecture.md: layers, modules, design rules.
- build.md: building, options, every environment variable.
- testing.md: test binaries, references, tolerances, sanitizers.
- benchmarks.md: measured performance, with OKLM against llama.cpp round by round (the current standing first).
- release-notes-1.0.1.md: what changed between 1.0.0 and 1.0.1, with the three changes of behaviour.
- release-notes-1.0.2.md: what changed between 1.0.1 and 1.0.2: the chats on a server that is not OKLM's, the packages published by CI, the build on Debian 13.
- release-notes-1.0.3.md: what changed between 1.0.2 and 1.0.3: the sources pass every job of CI (clang, the tools of Debian 13, the tests in a tree that is no git checkout).
- release-notes-1.0.0.md: what version 1.0.0 holds, what it promises to keep, its known limits.
- release-notes-0.1.0.md: the models trained from scratch and released as 0.1.0.
- roadmap.md: the milestones' state and what is left (small leftovers, known speed gaps, larger work, what is blocked or deferred).
- ROADMAP_DONE.md: what the roadmap planned and is finished, each item with its proof (the milestones' steps, RAG, the done blocks).
- kernel-autotune-plan.md: plan of the on-device kernel auto-tuning pass (on the roadmap).
- llm_training_datasets.md: training datasets considered for the training milestone, with sizes and licenses.
- compatibility.md: open-weight families, quant types and tokenizers that load.
Modules
- core.md: element types, tensors, arena, backends, CPU ops.
- convert.md:
oklm info,oklm convert,oklm quantize, and their verification against llama.cpp. - tokenizer.md: byte-level and SentencePiece BPE, files, pre-tokenizers, training, verification against Hugging Face.
- opencl.md: the OpenCL backend: device selection, memory, kernels, fallback, verification, speed.
- vulkan.md: the Vulkan backend: loading, device choice, buffer device addresses, shaders and pipelines, the groups of ops, the fallback, verification.
- training.md: the trainer (F32 forward
and backward passes, AdamW), checkpoints,
oklm train, verification against llm.c. - inference.md: model loading, KV cache,
Llama-family and GPT-2 forward passes, sampling, chat templates (Jinja
interpreter),
oklm run,oklm chat,oklm ppl, verification against llama.cpp. - moe.md: mixture-of-experts models: grouped
expert products, the expert cache and its streaming settings,
--n-cpu-moe, the device expert tier, their measurements. - observe.md: activations of a running
model: what each block writes, attention rows, the experts picked
(
--observe,/api/activations), and what those figures do and do not mean. - models.md:
oklm models: the catalogue of models offered for download (OKLM ships none), the pull, the licences, the checks. - memory-plan.md:
oklm plan,oklm plan --listand the memory warning ofrun,chatandserve. - cli.md: the command reference: every
oklmcommand and everyoklm-guitool with its synopsis (generated from the usage text), its options and the page that describes it. - commands.md: every slash command of the agent (generated from the registry).
- git.md: what the agent does in a git repository and what it asks first.
- processes.md: background jobs, monitors, services, language and MCP servers, integrations.
- harness.md: the agent harness: an overview of its parts and how they are tested.
- agent-loop.md: the agent loop, the
system prompt, backends and stream events, headless and library use, an
agent other programs attach to (
--listen), evals. - agent-tools.md: the built-in tools, skills, deferred tools and ToolSearch, sub-agents, git.
- agent-permissions.md:
permission rules and modes, the audit log and
oklm audit, the sandbox. - agent-settings.md: settings files,
oklm permissions, every setting, usage, the debug log and telemetry, memory files. - agent-ui.md: slash commands and the terminal UI.
- agent-sessions.md: context, compaction and sessions.
- agent-extensions.md: hooks, plugins, MCP servers.
- server.md: the HTTP API:
oklm serve, its endpoints, access rules, the dashboard and the scheduler. - rocm_spec.md: a ROCm/HIP training backend (hipBLASLt products, LoRA forward and backward, LoRA in the trainer and its files), with what OKLM already has; roadmap M15.
- mcp_training_spec.md: fine-tuning a LoRA adapter that drives an MCP tool server through the agent (schema-valid tool calls, multi-step workflows), on the general default base; roadmap M16.
- rag_spec.md: retrieval-augmented generation (embeddings, a local index, hybrid search, reranking, retrieval in run / chat / serve and the agent, tool selection by retrieval, corrective, agentic, graph and multimodal variants), with what OKLM already has; roadmap M17.
- rag.md: retrieval, the user guide: embeddings
(
oklm embed,oklm_embed(),/v1/embeddings), the models checked, the RAG evaluation and its results. - rag-design.md: how that specification maps onto OKLM (file layout, stage contracts, API, commands and endpoints, decisions, tests per step, the RAG evaluation; its K0, audit in gap-analysis.md).
- rag.md: retrieval: the chunkers, the index
(
oklm rag index,search,info), exact dense search, BM25 and hybrid fusion, measured. - serving_spec.md: vLLM-style serving (paged KV cache, continuous batching without fixed sequences, prefix caching across requests); to do, starting with an audit of what the server does (M14).
- serving-design.md: how that specification maps onto the model library and the server, its decisions, and today's measured baseline (its S0; audit in gap-analysis.md).
- gui.md:
oklm-gui, the native graphical tools in one program, a section a tool with its screenshots, worked examples, controls and options: the launcher, the chat window (its Models window, its context slider, connecting to a server, a listening agent or a session), the model manager, the server manager, the conversion tool, the model inspector, the training window, the sessions browser and the retrieval manager; how a tool is written; and the API's HTTP client. - multimodal.md: images, sounds and
speech: decoding, vision and audio encoders, Whisper,
oklm describe,oklm transcribe. - speech.md: speech synthesis: OuteTTS 0.2,
0.3 and WavTokenizer's decoder audited,
oklm speak, the comparison with llama.cpp, the limits, the other models looked at.
Formats
- formats/quant.md: byte layout of every block quant.
- formats/gguf.md: GGUF layout, reader validation, writer, shards.
- formats/safetensors.md: safetensors layout, validation, repositories.
Decisions
- adr/0001-language-and-dependencies.md
- adr/0002-weight-formats.md
- adr/0003-json-on-the-model-path.md
- adr/0004-pre-tokenizer-regex.md
- adr/0005-kv-cache-precision.md
- adr/0006-chat-templates.md
- adr/0007-video-containers.md
- adr/0008-training-checkpoints.md
- adr/0009-tool-calls-and-grammars.md
- adr/0010-permission-rules.md
- adr/0011-sandbox.md
- adr/0012-jobs.md
- adr/0013-k-quant-write-path.md
- adr/0014-nilorea-as-source-tree.md
- adr/0015-test-references.md
- adr/0016-cpu-worker-team.md
- adr/0017-opencl-backend.md
- adr/0018-server-api.md
- adr/0019-paged-kv-cache.md (accepted with amendments)
- adr/0020-lora-adapters.md
- adr/0021-adapters-beside-the-weights.md
- adr/0022-rag-index-format.md (proposed)
- adr/0023-vulkan-backend.md