Research: what OKLM builds on
This page summarizes the formats, reference implementations and model architectures OKLM re-implements, with the details that matter when writing the code. It is the long form of the "what exists today" survey (October 2026) and is updated when a module discovers something the survey missed.
Status: maintained.
Weight files
GGUF (ggml-org, v3)
Specification: https://github.com/ggml-org/ggml/blob/master/docs/gguf.md.
- Little-endian. Magic
GGUF, thenuint32 version(3; 2 is still readable, 1 is obsolete),uint64 n_tensors,uint64 n_kv. n_kvmetadata entries: a GGUF string key (uint64 length+ bytes, no terminator), auint32value type, the value. 13 value types: u8, i8, u16, i16, u32, i32, f32, bool, string, array, u64, i64, f64. Arrays carry an element type and auint64count and may nest (arrays of arrays of strings occur in tokenizer metadata).n_tensorstensor infos: name,uint32 n_dims,uint64 dims[n_dims](innermost first, like OKLM'sne[]),uint32 ggml_type,uint64 offsetrelative to the start of the data section.- Padding up to
general.alignment(default 32), then the data section; each tensor's offset is a multiple of the alignment. - Everything needed to run a model is inside:
general.architecture, the<arch>.*hyper-parameters,tokenizer.ggml.*(model, pre, tokens, scores, token types, merges, special ids),tokenizer.chat_template. - Sharded models (
-00001-of-0000N.gguf) usesplit.no,split.countandsplit.tensors.count.
ggml block quantization
Reference: ggml-common.h (layouts) and
ggml-quants.c (*_ref algorithms) in https://github.com/ggml-org/ggml.
The byte layout of every type OKLM handles is in formats/quant.md.
- Legacy quants: 32-element blocks with an f16 scale
(
Q8_0:w = d·q,Q4_0:w = d·(q−8)), plus an f16 min for the_1variants and a fifth bit plane forQ5_*.Q8_1storess = d·ΣqsoQ4_1/Q5_1dot products fold the min in without a second pass. - K-quants: 256-element super-blocks split into 8 or 16 sub-blocks
whose scales (and mins) are themselves quantized to 4 or 6 bits against
f16 super-scales. Published GGUFs are mostly
Q4_K_M(Q4_K with Q6_K for some tensors),Q5_K_MandQ6_K, so reading K-quants is required to load the models people have. - Dot products quantize the activations to a "vec_dot type" first
(
Q8_0,Q8_1orQ8_K), so the inner loop is an integer dot product.
safetensors
Reference: https://github.com/huggingface/safetensors (README, "Format").
uint64 N (little-endian), then N bytes of
UTF-8 JSON
{"name": {"dtype": "F16", "shape": [...], "data_offsets": [begin, end]}, "__metadata__": {...}},
then the raw buffer. The header must start with {, offsets
must not overlap, and the header is capped at 100 MB. Models come with
config.json (architecture hyper-parameters),
tokenizer.json (Hugging Face tokenizers: pre-tokenizer,
model.vocab, model.merges,
added_tokens) and, when sharded,
model.safetensors.index.json.
Reference implementations (read, never copied)
- llama2.c (https://github.com/karpathy/llama2.c): Llama 2 inference in about 700 lines of C, fp32, fixed architecture. Shows how small a correct forward pass is.
- llm.c (https://github.com/karpathy/llm.c):
GPT-2 training in about 1000 lines of C. Every layer has
*_forwardand*_backward, parameters, gradients and activations live in one preallocated arena, AdamW, an int32 token-stream loader. OKLM's training follows this structure. - llama.cpp (https://github.com/ggml-org/llama.cpp):
GGUF loading, quantized
mul_mat, KV cache, grouped-query attention, sampling, chat templates,mmprojvision files, an OpenCL backend (Q4_0-optimized). - whisper.cpp (https://github.com/ggml-org/whisper.cpp): log-mel spectrogram (Hann window, 400-point FFT, hop 160, 80 mel bins, 30 s chunks of 80×3000), two conv1d + GELU, sinusoidal positions, encoder blocks, a decoder with cross-attention.
Model architectures
Modern decoder block (Llama 2/3, Mistral, Qwen, Gemma, Phi)
Pre-RMSNorm → Q, K, V projections → RoPE on Q and K → grouped-query
causal attention over the KV cache → residual → pre-RMSNorm → SwiGLU
feed-forward down(silu(gate(x)) * up(x)) → residual. No
biases in Llama; Qwen2 adds QKV biases, Qwen3 adds per-head Q/K norms,
Mistral a sliding window, Gemma GeGLU, embedding scaling by √d, extra
post-norms and logit soft-capping.
RoPE comes in two pair layouts: adjacent pairs
(2i, 2i+1) ("normal", GPT-J, and Llama weights as stored in
GGUF after llama.cpp's Q/K permutation) and halves
(i, i + d/2) ("NeoX": Qwen, Phi, Gemma, and Llama weights
as stored in Hugging Face safetensors). Long-context variants change the
angles: linear scaling divides positions, YaRN mixes interpolated and
extrapolated frequencies over a ramp and scales the magnitude, Llama 3
divides each frequency by a per-dimension factor. OKLM's
oklm_rope() takes the same parameter set as ggml's
rope_ext so all of these are covered.
GPT-2
LayerNorm, learned position embeddings, GELU MLP, biases everywhere, fused QKV. Supported second because llm.c gives a full training reference.
Vision (LLaVA pattern)
A frozen CLIP or SigLIP ViT (224 or 336 px, patch 14, 256 or 576
patch tokens), a two-layer MLP projector into the LLM embedding space,
visual tokens inserted at an <image> placeholder.
llama.cpp stores the ViT and projector in a separate
mmproj-*.gguf (architecture clip, keys
clip.has_vision_encoder, clip.projector_type,
clip.vision.*).
OpenCL matrix products
Cedric Nugteren's SGEMM tutorial (https://cnugteren.github.io/tutorial/pages/page1.html) is the roadmap: naive kernel → local-memory tiles → more work per thread → wider loads → transposed operand → padding for arbitrary sizes → prefetching. Decode is memory-bound, so quantized-weight kernels that dequantize in registers matter more than peak FLOPS.
Tokenizers
- Byte-level BPE (GPT-2: 50 257 tokens, Llama 3: 128 256, tiktoken
cl100k/o200k): regex pre-tokenization, a byte→unicode table, greedy merging by rank. - SentencePiece-BPE (Llama 2: 32 000):
▁marks spaces, merges ordered by score, unknown bytes fall back to<0xNN>tokens. - Training BPE: count adjacent pairs, merge the most frequent, repeat up to the target vocabulary size.
nilorea-library
Apache-2.0, C17, Linux / MinGW / Android (https://git.nilorea.net/gullradriel/nilorea-library).
OKLM uses it for logging (n_log), containers
(n_list, n_hash, n_str), threads
(n_thread_pool; the CPU backend's worker team, first
nilorea's N_PARALLEL_TEAM, is now OKLM's own), files, time,
JSON (vendored cJSON), compression, networking (TCP/TLS/WebSocket/SSE),
the HTTP server and router, and the Allegro 5 n_gui widget
system.
Agent file conventions
A skill is a directory with SKILL.md (YAML front matter
name, description, optional
allowed-tools) plus optional scripts/,
references/, assets/; a sub-agent is
agents/<name>.md with front matter name,
description, tools, model and the
system prompt as body. Only names and descriptions are loaded at
start-up; bodies load on activation. Specification: https://agentskills.io/specification.