ADR-0017: OpenCL backend design
Records how the OpenCL backend fits into OKLM's tensor engine.
Status: accepted (2026-10-30).
Context
Every op already dispatches on its destination's backend
(oklm_tensor.backend), and the forward passes are written
once against the op API. A GPU backend has to keep that: no second copy
of the model code, views working as on the CPU, and a binary that builds
and runs where OpenCL is absent.
Decision
- Runtime loading. OpenCL types and entry points are
declared in OKLM and the ICD loader is opened with
dlopen; no header or library is needed to build, and the backend is probed only whenopenclorautois requested. - Fake device addresses. Each device buffer owns a range of addresses that are never dereferenced on the host. Tensors and views on the device look like host tensors (pointer plus strides), and the backend maps an address back to a buffer and offset at launch. Kernels take buffer, offset, sizes and strides, so any view works without copies.
- CPU fallback for everything. An op without a suitable kernel runs on the CPU over host copies of the byte ranges it touches. Every op works on the device from the start; kernels are added for speed, each one tested against the CPU.
- Float activations in matrix products. Kernels dequantize the weights and multiply by F32 activations instead of rounding activations to Q8 as the CPU does. This removes the activation-rounding noise (quantized models match the float64 reference to 1e-4), at the price of results that differ from the CPU's by that noise.
- Placement. Weights copied once per model to one device buffer, KV cache and activations per context on the device, token and position embeddings left on the host (looked up through the fallback).
- Defaults. The library default stays the CPU
(reproducible numbers for embedders and tests); the
oklmcommands useauto.
Consequences
The forward passes did not change beyond reading weights through a per-context table. Ops with only a CPU implementation still run on an OpenCL context, at the cost of transfers. Splitting layers between CPU and GPU and streaming models larger than device memory need cross-backend copies, which the design leaves for later.