Kernel tuning: results
Status: maintained.
Intel Core i7-11700K (8 cores, AVX-512 VNNI), 8 threads, 2026-10-31,
oklm tune -m <file> --backend cpu (512-token prompt,
60 s budget):
| Model | Prompt, defaults | Candidates | Rejected | Kept |
|---|---|---|---|---|
| Llama-3.2-1B Q4_K_M | 938.6 ms | 11 | 0 | the defaults (no candidate 3 % faster) |
On this machine the defaults are already the best work split: manual
runs of matmul_rows 8 to 64 moved Llama-3.2-1B's prompt
speed by about 1 % (553 to 559 t/s), within noise. The tuner's value is
on other CPUs (cache sizes, core counts) and other models.
Output equivalence: every candidate gave the default logits bit for
bit (none rejected), the settings kept are checked on a second prompt,
and test_tune checks that a context gives the same logits
with any work split. With --backend opencl and every layer
on the device, the tuner times the defaults and searches nothing (94.9
ms for the same prompt).
The expert cache's streaming settings (loaders, read-ahead,
O_DIRECT) are tuned the same way by
oklm tune --expert-cache GB, with their own cache entries
beside these: see moe.md.