OKLMdocumentation v1.0.3
On this page

Research: what OKLM builds on

This page summarizes the formats, reference implementations and model architectures OKLM re-implements, with the details that matter when writing the code. It is the long form of the "what exists today" survey (October 2026) and is updated when a module discovers something the survey missed.

Status: maintained.

Weight files

GGUF (ggml-org, v3)

Specification: https://github.com/ggml-org/ggml/blob/master/docs/gguf.md.

  • Little-endian. Magic GGUF, then uint32 version (3; 2 is still readable, 1 is obsolete), uint64 n_tensors, uint64 n_kv.
  • n_kv metadata entries: a GGUF string key (uint64 length + bytes, no terminator), a uint32 value type, the value. 13 value types: u8, i8, u16, i16, u32, i32, f32, bool, string, array, u64, i64, f64. Arrays carry an element type and a uint64 count and may nest (arrays of arrays of strings occur in tokenizer metadata).
  • n_tensors tensor infos: name, uint32 n_dims, uint64 dims[n_dims] (innermost first, like OKLM's ne[]), uint32 ggml_type, uint64 offset relative to the start of the data section.
  • Padding up to general.alignment (default 32), then the data section; each tensor's offset is a multiple of the alignment.
  • Everything needed to run a model is inside: general.architecture, the <arch>.* hyper-parameters, tokenizer.ggml.* (model, pre, tokens, scores, token types, merges, special ids), tokenizer.chat_template.
  • Sharded models (-00001-of-0000N.gguf) use split.no, split.count and split.tensors.count.

ggml block quantization

Reference: ggml-common.h (layouts) and ggml-quants.c (*_ref algorithms) in https://github.com/ggml-org/ggml. The byte layout of every type OKLM handles is in formats/quant.md.

  • Legacy quants: 32-element blocks with an f16 scale (Q8_0: w = d·q, Q4_0: w = d·(q−8)), plus an f16 min for the _1 variants and a fifth bit plane for Q5_*. Q8_1 stores s = d·Σq so Q4_1/Q5_1 dot products fold the min in without a second pass.
  • K-quants: 256-element super-blocks split into 8 or 16 sub-blocks whose scales (and mins) are themselves quantized to 4 or 6 bits against f16 super-scales. Published GGUFs are mostly Q4_K_M (Q4_K with Q6_K for some tensors), Q5_K_M and Q6_K, so reading K-quants is required to load the models people have.
  • Dot products quantize the activations to a "vec_dot type" first (Q8_0, Q8_1 or Q8_K), so the inner loop is an integer dot product.

safetensors

Reference: https://github.com/huggingface/safetensors (README, "Format").

uint64 N (little-endian), then N bytes of UTF-8 JSON {"name": {"dtype": "F16", "shape": [...], "data_offsets": [begin, end]}, "__metadata__": {...}}, then the raw buffer. The header must start with {, offsets must not overlap, and the header is capped at 100 MB. Models come with config.json (architecture hyper-parameters), tokenizer.json (Hugging Face tokenizers: pre-tokenizer, model.vocab, model.merges, added_tokens) and, when sharded, model.safetensors.index.json.

Reference implementations (read, never copied)

  • llama2.c (https://github.com/karpathy/llama2.c): Llama 2 inference in about 700 lines of C, fp32, fixed architecture. Shows how small a correct forward pass is.
  • llm.c (https://github.com/karpathy/llm.c): GPT-2 training in about 1000 lines of C. Every layer has *_forward and *_backward, parameters, gradients and activations live in one preallocated arena, AdamW, an int32 token-stream loader. OKLM's training follows this structure.
  • llama.cpp (https://github.com/ggml-org/llama.cpp): GGUF loading, quantized mul_mat, KV cache, grouped-query attention, sampling, chat templates, mmproj vision files, an OpenCL backend (Q4_0-optimized).
  • whisper.cpp (https://github.com/ggml-org/whisper.cpp): log-mel spectrogram (Hann window, 400-point FFT, hop 160, 80 mel bins, 30 s chunks of 80×3000), two conv1d + GELU, sinusoidal positions, encoder blocks, a decoder with cross-attention.

Model architectures

Modern decoder block (Llama 2/3, Mistral, Qwen, Gemma, Phi)

Pre-RMSNorm → Q, K, V projections → RoPE on Q and K → grouped-query causal attention over the KV cache → residual → pre-RMSNorm → SwiGLU feed-forward down(silu(gate(x)) * up(x)) → residual. No biases in Llama; Qwen2 adds QKV biases, Qwen3 adds per-head Q/K norms, Mistral a sliding window, Gemma GeGLU, embedding scaling by √d, extra post-norms and logit soft-capping.

RoPE comes in two pair layouts: adjacent pairs (2i, 2i+1) ("normal", GPT-J, and Llama weights as stored in GGUF after llama.cpp's Q/K permutation) and halves (i, i + d/2) ("NeoX": Qwen, Phi, Gemma, and Llama weights as stored in Hugging Face safetensors). Long-context variants change the angles: linear scaling divides positions, YaRN mixes interpolated and extrapolated frequencies over a ramp and scales the magnitude, Llama 3 divides each frequency by a per-dimension factor. OKLM's oklm_rope() takes the same parameter set as ggml's rope_ext so all of these are covered.

GPT-2

LayerNorm, learned position embeddings, GELU MLP, biases everywhere, fused QKV. Supported second because llm.c gives a full training reference.

Vision (LLaVA pattern)

A frozen CLIP or SigLIP ViT (224 or 336 px, patch 14, 256 or 576 patch tokens), a two-layer MLP projector into the LLM embedding space, visual tokens inserted at an <image> placeholder. llama.cpp stores the ViT and projector in a separate mmproj-*.gguf (architecture clip, keys clip.has_vision_encoder, clip.projector_type, clip.vision.*).

OpenCL matrix products

Cedric Nugteren's SGEMM tutorial (https://cnugteren.github.io/tutorial/pages/page1.html) is the roadmap: naive kernel → local-memory tiles → more work per thread → wider loads → transposed operand → padding for arbitrary sizes → prefetching. Decode is memory-bound, so quantized-weight kernels that dequantize in registers matter more than peak FLOPS.

Tokenizers

  • Byte-level BPE (GPT-2: 50 257 tokens, Llama 3: 128 256, tiktoken cl100k/o200k): regex pre-tokenization, a byte→unicode table, greedy merging by rank.
  • SentencePiece-BPE (Llama 2: 32 000): ▁ marks spaces, merges ordered by score, unknown bytes fall back to <0xNN> tokens.
  • Training BPE: count adjacent pairs, merge the most frequent, repeat up to the target vocabulary size.

nilorea-library

Apache-2.0, C17, Linux / MinGW / Android (https://git.nilorea.net/gullradriel/nilorea-library). OKLM uses it for logging (n_log), containers (n_list, n_hash, n_str), threads (n_thread_pool; the CPU backend's worker team, first nilorea's N_PARALLEL_TEAM, is now OKLM's own), files, time, JSON (vendored cJSON), compression, networking (TCP/TLS/WebSocket/SSE), the HTTP server and router, and the Allegro 5 n_gui widget system.

Agent file conventions

A skill is a directory with SKILL.md (YAML front matter name, description, optional allowed-tools) plus optional scripts/, references/, assets/; a sub-agent is agents/<name>.md with front matter name, description, tools, model and the system prompt as body. Only names and descriptions are loaded at start-up; bodies load on activation. Specification: https://agentskills.io/specification.