OKLMdocumentation v1.0.3
On this page

Kernel tuning: results

Status: maintained.

Intel Core i7-11700K (8 cores, AVX-512 VNNI), 8 threads, 2026-10-31, oklm tune -m <file> --backend cpu (512-token prompt, 60 s budget):

Model Prompt, defaults Candidates Rejected Kept
Llama-3.2-1B Q4_K_M 938.6 ms 11 0 the defaults (no candidate 3 % faster)

On this machine the defaults are already the best work split: manual runs of matmul_rows 8 to 64 moved Llama-3.2-1B's prompt speed by about 1 % (553 to 559 t/s), within noise. The tuner's value is on other CPUs (cache sizes, core counts) and other models.

Output equivalence: every candidate gave the default logits bit for bit (none rejected), the settings kept are checked on a second prompt, and test_tune checks that a context gives the same logits with any work split. With --backend opencl and every layer on the device, the tuner times the defaults and searches nothing (94.9 ms for the same prompt).

The expert cache's streaming settings (loaders, read-ahead, O_DIRECT) are tuned the same way by oklm tune --expert-cache GB, with their own cache entries beside these: see moe.md.