Kernel tuning: parameters
What the tuner may change, and what it may not (kernel-autotune-plan.md).
Status: maintained.
Searched
oklm_cpu_tuning (include/oklm/core/cpu.h),
process-wide like the thread count, set by
oklm_cpu_set_tuning():
| Setting | Default | Values tried | Kernel |
|---|---|---|---|
matmul_rows |
16 | 8, 16, 32, 64, 128 | weight rows per task of matrix products of several activation rows
(matmul_cpu.c; packed weights in groups of 8) |
attn_rows |
32 | 8, 16, 24, 32 | query rows per attention tile (attention_cpu.c) |
attn_keys |
64 | 16, 32, 64, 128, 256 | keys converted to float at a time by tiled attention |
Each is read at the start of the op; the defaults reproduce the code before the settings existed.
Not searched
- The arithmetic of the CPU kernels: their results must stay those of the scalar kernels, so tile shapes that reorder a sum are out.
- The OpenCL thresholds (
OKLM_CL_GEMM_TILES,OKLM_CL_GEMM_SPLIT,OKLM_CL_ATT_BLOCKS,OKLM_CL_ATT_KEYS) and work-group sizes: each one picks another kernel or another reduction order, which changes logits, so the bitwise check would reject every candidate. They stay manual knobs (opencl.md); a context with every layer on the device has nothing for the tuner to change.
Dispatch
There is no dispatch table to add: the kernels read the settings when
they run, and oklm_context_new() applies the cached ones
(oklm_tune_load()) unless OKLM_TUNE=0.