| llama-quantize, gguf-py | ||
| Hugging Face tokenizers | ||
| jinja2 | ||
| llama.cpp | ||
| llm.c + PyTorch | ||
| llama.cpp mtmd | ||
| llama.cpp | ||
| llama.cpp | ||
oklm tune -m model.gguf
--backend vulkan
--backend opencl
train --backend rocm
| pp 1 th | pp 4 | pp 16 | tg 1 th | tg 4 | tg 16 | |
|---|---|---|---|---|---|---|
| SmolLM2-135M Q4_K_M | 1.29 | 1.35 | 1.52 | 1.56 | 1.09 | 1.16 |
| Llama-3.2-1B Q4_K_M | 1.08 | 1.09 | 1.20 | 1.16 | 1.00 | 1.09 |
| Qwen3-8B Q8_0 | 1.76 | 1.76 | 1.80 | 1.22 | 1.00 | 1.08 |
| OLMoE-1B-7B Q4_K_M | 1.78 | 1.75 | 1.94 | 1.17 | 0.99 | 1.00 |
| GLM-4.7-Flash Q4_K_M | 1.29 | 1.33 | 1.42 | 1.16 | 1.02 | 0.98 |
| pp512 | tg128 | |
|---|---|---|
| SmolLM2-135M Q8_0 | 1.37 | 1.17 |
| SmolLM2-135M Q4_K_M | 1.15 | 1.04 |
| Llama-3.2-1B Q4_K_M | 0.84 | 0.95 |
| Qwen3-8B Q4_K_M | 0.86 | 0.90 |
| Qwen3-8B Q8_0 | 1.65 | 0.92 |
| Granite-3.1-1B-A400M | 1.06 | 1.09 |
| OLMoE-1B-7B | 0.85 | 0.88 |
| GLM-4.7-Flash | 1.04 | 0.89 |
| prompt | ||
|---|---|---|
| RX 7900 XT, Llama-3.2-1B, Vulkan | 16 235 | 509 |
| RX 7900 XT, Llama-3.2-1B, OpenCL | 14 861 | 403 |
| RX 7900 XT, Qwen3-8B, Vulkan | 2 663 | 112 |
| RX 7900 XT, Qwen3-8B, OpenCL | 2 747 | 94.6 |
| UHD 750, Llama-3.2-1B, Vulkan | 55 | 22.1 |
| UHD 750, Llama-3.2-1B, OpenCL | 8.9 | 5.4 |
oklm run | chat | ppl
oklm serve -m model.gguf
oklm-gui --tool NAME
oklm describe | transcribe | speak | rag
oklm agent -m qwen3-8b.gguf
oklm train | rl
| OuteTTS 0.2, CPU | 6.6 s | 0.51 |
| OuteTTS 0.2, OpenCL | 0.9 s | 3.7 |
| OuteTTS 0.2, Vulkan | 0.7 s | 4.8 |
| Soprano 1.1, CPU | 0.2 s | 12.6 - 13.1 |
| Soprano 1.1, OpenCL | 0.06 s | 39 |
| Soprano 1.1, Vulkan | 0.06 s | 44 - 47 |
| SciFact | docs OKLM | |
|---|---|---|
| 71.27 | 60.48 | |
| BM25 | 67.41 | 64.53 |
| 71.29 | 69.78 | |
| 73.04 | 70.61 |
train -f text.txt --arch llama
train -m model.gguf -f text.txt
train --resume ckpt-000600.gguf
train -m m.gguf --lora 8 --quantized-base
train --chat-data conv.jsonl
rl -m model.gguf -f tasks.jsonl
| F32 | 3.1329 |
| 3.2365 | |
| 3.1347 |
| llm.c | 4.10778 |
| OKLM | 4.10773 |
| OKLM-Nano | 5.3 M | Tiny Shakespeare | - | 18 min (CPU) | 3.99 |
| OKLM-Micro | 33.6 M | SmolTalk | 1 h + 35 min | 1.39 / 2.93 | |
| OKLM-Small | 88.1 M | SmolTalk | 6.7 h + 35 min | 3.51 / 2.37 |
| backend | LoRA all | LoRA qkvo | |
|---|---|---|---|
| opencl | 156 ms | 140 ms | 119 ms |
| rocm | 110 ms | 102 ms | 91 ms |
| cpu | - | 3.3 s | - |
oklm imatrix ; oklm quantize --imatrix
oklm lora-merge base.gguf ad.gguf out.gguf
oklm abliterate ; oklm refusals --ref
oklm ppl ; oklm info -t ; oklm tune
| Qwen3-8B | |
| Qwen3-4B | |
| Qwen3-0.6B |