OKLMdocumentation v1.0.3
On this page

Kernel tuning: tolerances

How the tuner decides that a candidate gives "identical output" (kernel-autotune-plan.md, harness and oracle).

Status: maintained.

Both backends are deterministic (00-codebase-map.md): the default configuration run twice gives the same bits, so the run-to-run noise is zero and the tolerance is zero too. The check is bitwise, for every setting:

  • the oracle is the default configuration's output: the logits of the last token of a fixed 512-token pseudo-random prompt, from an empty cache (src/model/tune.c);
  • a candidate is kept only when its logits are the same bytes (memcmp), and the winner is checked again on a second prompt;
  • a candidate faster by less than 3 % is not kept (timing noise), so the defaults stay unless a setting clearly helps.

The settings searched (oklm_cpu_tuning) only split the CPU kernels' work: each output keeps the order of its sums whatever their values, so they pass the check by construction. test_matmul and test_attention check that bit for bit for several splits, and test_tune checks that a candidate with altered logits is rejected.