ADR-0006: Chat templates
Records how OKLM turns a conversation into the prompt text a chat model expects.
Status: accepted (2026-10-29).
Context
Every chat model ships its prompt format as a Jinja template
(tokenizer.chat_template in GGUF,
chat_template in tokenizer_config.json),
written for Python's jinja2 in the environment Hugging Face transformers
sets up for apply_chat_template(). Templates go well beyond
simple substitution: Qwen3 walks the messages backwards with a namespace
and splits <think> blocks with string methods, Llama
3.2 calls strftime_now and tojson(indent=4),
Mistral filters with selectattr, Gemma and Mistral reject
malformed conversations with raise_exception. Recognizing a
handful of formats by their markers and rendering them with built-in
code works for the common cases but cannot follow a template's own logic
(system prompt placement, tool calls, thinking blocks, dates).
Decision
- A Jinja interpreter written for OKLM
(
src/model/jinja.c, API ininclude/oklm/model/jinja.h): lexer with jinja2's whitespace rules (trim_blocks,lstrip_blocks,-/+markers, final newline dropped), a parser to a syntax tree, and a tree-walking evaluator with Python value semantics (printing, equality, slicing, integer and float arithmetic, undefined values) and jinja2's scoping (a fresh scope per loop iteration,namespace()to carry state out). - It covers what published templates use: all statements but template
inheritance and includes, the common filters, tests and Python string /
dict methods,
range,namespace,dict,raise_exception,strftime_now,{% generation %}, and Hugging Face'stojson(json.dumpswithensure_ascii=False). - Variables are passed as JSON, so a caller can give a template
anything Hugging Face would (messages with tool calls,
tools,documents, extra flags such asenable_thinking). The interpreter reads that JSON itself rather than throughn_json(ADR-0003): templates print1and1.0differently, as Python does, and cJSON does not keep integers apart from floats. - Templates come from model files and are untrusted: nesting depth,
evaluation steps, memory and output size are bounded, every error is
reported, and the interpreter is fuzzed
(
fuzz/fuzz_jinja.c). - The built-in renderers stay as the fallback when a model has no
template or one the interpreter rejects
(
oklm_chat_template_new()logs why). - A format chosen by name (
--chat-template NAME, a catalogue entry) is itself a Jinja template compiled in, put in place of the file's: one path renders it, and what reads the template's text sees it. - As llama.cpp does, a leading BOS text produced by the template is removed when the tokenizer adds BOS itself.
Verification
tests/model/test_jinja.c renders, byte for byte like
jinja2 in Hugging Face's environment (tools/jinja_ref.py):
157 committed cases covering the language and its error paths, and the
published GLM-4.5 / 4.6 and GLM-4.7 templates (copied from llama.cpp's
models/templates/ into tests/fixtures/jinja/)
on a conversation with tools, calls, results and reasoning and on a lone
question, thinking left to the template, on, off, and kept
(clear_thinking: false); in make test-public
the published templates of 11 families (SmolLM2, Qwen2.5, Qwen3, Llama
3.2, TinyLlama, Mistral, Phi-3.5, DeepSeek-R1, Gemma 3, Granite 3.3,
Hermes 3) on six conversations each, with and without the generation
prompt.
Consequences
Any model whose template stays within the supported language renders
exactly as with transformers, without a per-model table. Recursive
loops, loop.cycle() and call blocks are
supported; template inheritance, include and
import are rejected (they need a template loader, which
chat templates never have) and such a template falls back to the
built-in formats. Case conversions follow Python's str methods over all
of Unicode, including the final-sigma rule; jinja2's title
filter (which is not str.title()) is reproduced as
such.