OKLMdocumentation v1.0.3
On this page

OKLM documentation

OKLM is a self-contained LLMOps framework: a dependency-free LLM runtime and training stack in C17. One source tree loads, converts, quantizes, runs, trains, fine-tunes, serves and drives models as an agent, with no external library touching the model (the optional ones serve the edges: TLS, the GUI window, video decoding).

Table of contents of the OKLM documentation. Every page starts with a one-paragraph purpose and a status line.

Status: maintained.

Guides

Start here: pages written to be read through, which point to the references below for the details.

  • how-it-works.md: a guided tour for newcomers to LLM internals: model files, quantization, an end-to-end quantization run, memory, backends, generation, the server, the agent, training, multimodal, refusal analysis and the developer tooling, how and why.
  • user-guide.md: using oklm agent day to day.
  • extending.md: adding to the agent: skills, commands, sub-agent types, hooks, MCP servers, plugins, services, the library API.

Project

  • research.md: the state of the art OKLM builds on: formats, reference implementations, model architectures.
  • architecture.md: layers, modules, design rules.
  • build.md: building, options, every environment variable.
  • testing.md: test binaries, references, tolerances, sanitizers.
  • benchmarks.md: measured performance, with OKLM against llama.cpp round by round (the current standing first).
  • release-notes-1.0.1.md: what changed between 1.0.0 and 1.0.1, with the three changes of behaviour.
  • release-notes-1.0.2.md: what changed between 1.0.1 and 1.0.2: the chats on a server that is not OKLM's, the packages published by CI, the build on Debian 13.
  • release-notes-1.0.3.md: what changed between 1.0.2 and 1.0.3: the sources pass every job of CI (clang, the tools of Debian 13, the tests in a tree that is no git checkout).
  • release-notes-1.0.0.md: what version 1.0.0 holds, what it promises to keep, its known limits.
  • release-notes-0.1.0.md: the models trained from scratch and released as 0.1.0.
  • roadmap.md: the milestones' state and what is left (small leftovers, known speed gaps, larger work, what is blocked or deferred).
  • ROADMAP_DONE.md: what the roadmap planned and is finished, each item with its proof (the milestones' steps, RAG, the done blocks).
  • kernel-autotune-plan.md: plan of the on-device kernel auto-tuning pass (on the roadmap).
  • llm_training_datasets.md: training datasets considered for the training milestone, with sizes and licenses.
  • compatibility.md: open-weight families, quant types and tokenizers that load.

Modules

  • core.md: element types, tensors, arena, backends, CPU ops.
  • convert.md: oklm info, oklm convert, oklm quantize, and their verification against llama.cpp.
  • tokenizer.md: byte-level and SentencePiece BPE, files, pre-tokenizers, training, verification against Hugging Face.
  • opencl.md: the OpenCL backend: device selection, memory, kernels, fallback, verification, speed.
  • vulkan.md: the Vulkan backend: loading, device choice, buffer device addresses, shaders and pipelines, the groups of ops, the fallback, verification.
  • training.md: the trainer (F32 forward and backward passes, AdamW), checkpoints, oklm train, verification against llm.c.
  • inference.md: model loading, KV cache, Llama-family and GPT-2 forward passes, sampling, chat templates (Jinja interpreter), oklm run, oklm chat, oklm ppl, verification against llama.cpp.
  • moe.md: mixture-of-experts models: grouped expert products, the expert cache and its streaming settings, --n-cpu-moe, the device expert tier, their measurements.
  • observe.md: activations of a running model: what each block writes, attention rows, the experts picked (--observe, /api/activations), and what those figures do and do not mean.
  • models.md: oklm models: the catalogue of models offered for download (OKLM ships none), the pull, the licences, the checks.
  • memory-plan.md: oklm plan, oklm plan --list and the memory warning of run, chat and serve.
  • cli.md: the command reference: every oklm command and every oklm-gui tool with its synopsis (generated from the usage text), its options and the page that describes it.
  • commands.md: every slash command of the agent (generated from the registry).
  • git.md: what the agent does in a git repository and what it asks first.
  • processes.md: background jobs, monitors, services, language and MCP servers, integrations.
  • harness.md: the agent harness: an overview of its parts and how they are tested.
  • agent-loop.md: the agent loop, the system prompt, backends and stream events, headless and library use, an agent other programs attach to (--listen), evals.
  • agent-tools.md: the built-in tools, skills, deferred tools and ToolSearch, sub-agents, git.
  • agent-permissions.md: permission rules and modes, the audit log and oklm audit, the sandbox.
  • agent-settings.md: settings files, oklm permissions, every setting, usage, the debug log and telemetry, memory files.
  • agent-ui.md: slash commands and the terminal UI.
  • agent-sessions.md: context, compaction and sessions.
  • agent-extensions.md: hooks, plugins, MCP servers.
  • server.md: the HTTP API: oklm serve, its endpoints, access rules, the dashboard and the scheduler.
  • rocm_spec.md: a ROCm/HIP training backend (hipBLASLt products, LoRA forward and backward, LoRA in the trainer and its files), with what OKLM already has; roadmap M15.
  • mcp_training_spec.md: fine-tuning a LoRA adapter that drives an MCP tool server through the agent (schema-valid tool calls, multi-step workflows), on the general default base; roadmap M16.
  • rag_spec.md: retrieval-augmented generation (embeddings, a local index, hybrid search, reranking, retrieval in run / chat / serve and the agent, tool selection by retrieval, corrective, agentic, graph and multimodal variants), with what OKLM already has; roadmap M17.
  • rag.md: retrieval, the user guide: embeddings (oklm embed, oklm_embed(), /v1/embeddings), the models checked, the RAG evaluation and its results.
  • rag-design.md: how that specification maps onto OKLM (file layout, stage contracts, API, commands and endpoints, decisions, tests per step, the RAG evaluation; its K0, audit in gap-analysis.md).
  • rag.md: retrieval: the chunkers, the index (oklm rag index, search, info), exact dense search, BM25 and hybrid fusion, measured.
  • serving_spec.md: vLLM-style serving (paged KV cache, continuous batching without fixed sequences, prefix caching across requests); to do, starting with an audit of what the server does (M14).
  • serving-design.md: how that specification maps onto the model library and the server, its decisions, and today's measured baseline (its S0; audit in gap-analysis.md).
  • gui.md: oklm-gui, the native graphical tools in one program, a section a tool with its screenshots, worked examples, controls and options: the launcher, the chat window (its Models window, its context slider, connecting to a server, a listening agent or a session), the model manager, the server manager, the conversion tool, the model inspector, the training window, the sessions browser and the retrieval manager; how a tool is written; and the API's HTTP client.
  • multimodal.md: images, sounds and speech: decoding, vision and audio encoders, Whisper, oklm describe, oklm transcribe.
  • speech.md: speech synthesis: OuteTTS 0.2, 0.3 and WavTokenizer's decoder audited, oklm speak, the comparison with llama.cpp, the limits, the other models looked at.

Formats

Decisions