OKLMdocumentation v1.0.3
On this page

Kernel tuning: parameters

What the tuner may change, and what it may not (kernel-autotune-plan.md).

Status: maintained.

Searched

oklm_cpu_tuning (include/oklm/core/cpu.h), process-wide like the thread count, set by oklm_cpu_set_tuning():

Setting Default Values tried Kernel
matmul_rows 16 8, 16, 32, 64, 128 weight rows per task of matrix products of several activation rows (matmul_cpu.c; packed weights in groups of 8)
attn_rows 32 8, 16, 24, 32 query rows per attention tile (attention_cpu.c)
attn_keys 64 16, 32, 64, 128, 256 keys converted to float at a time by tiled attention

Each is read at the start of the op; the defaults reproduce the code before the settings existed.

Not searched

  • The arithmetic of the CPU kernels: their results must stay those of the scalar kernels, so tile shapes that reorder a sum are out.
  • The OpenCL thresholds (OKLM_CL_GEMM_TILES, OKLM_CL_GEMM_SPLIT, OKLM_CL_ATT_BLOCKS, OKLM_CL_ATT_KEYS) and work-group sizes: each one picks another kernel or another reduction order, which changes logits, so the bitwise check would reject every candidate. They stay manual knobs (opencl.md); a context with every layer on the device has nothing for the tuner to change.

Dispatch

There is no dispatch table to add: the kernels read the settings when they run, and oklm_context_new() applies the cached ones (oklm_tune_load()) unless OKLM_TUNE=0.