OKLMdocumentation v1.0.3
On this page

ADR-0006: Chat templates

Records how OKLM turns a conversation into the prompt text a chat model expects.

Status: accepted (2026-10-29).

Context

Every chat model ships its prompt format as a Jinja template (tokenizer.chat_template in GGUF, chat_template in tokenizer_config.json), written for Python's jinja2 in the environment Hugging Face transformers sets up for apply_chat_template(). Templates go well beyond simple substitution: Qwen3 walks the messages backwards with a namespace and splits <think> blocks with string methods, Llama 3.2 calls strftime_now and tojson(indent=4), Mistral filters with selectattr, Gemma and Mistral reject malformed conversations with raise_exception. Recognizing a handful of formats by their markers and rendering them with built-in code works for the common cases but cannot follow a template's own logic (system prompt placement, tool calls, thinking blocks, dates).

Decision

  • A Jinja interpreter written for OKLM (src/model/jinja.c, API in include/oklm/model/jinja.h): lexer with jinja2's whitespace rules (trim_blocks, lstrip_blocks, - / + markers, final newline dropped), a parser to a syntax tree, and a tree-walking evaluator with Python value semantics (printing, equality, slicing, integer and float arithmetic, undefined values) and jinja2's scoping (a fresh scope per loop iteration, namespace() to carry state out).
  • It covers what published templates use: all statements but template inheritance and includes, the common filters, tests and Python string / dict methods, range, namespace, dict, raise_exception, strftime_now, {% generation %}, and Hugging Face's tojson (json.dumps with ensure_ascii=False).
  • Variables are passed as JSON, so a caller can give a template anything Hugging Face would (messages with tool calls, tools, documents, extra flags such as enable_thinking). The interpreter reads that JSON itself rather than through n_json (ADR-0003): templates print 1 and 1.0 differently, as Python does, and cJSON does not keep integers apart from floats.
  • Templates come from model files and are untrusted: nesting depth, evaluation steps, memory and output size are bounded, every error is reported, and the interpreter is fuzzed (fuzz/fuzz_jinja.c).
  • The built-in renderers stay as the fallback when a model has no template or one the interpreter rejects (oklm_chat_template_new() logs why).
  • A format chosen by name (--chat-template NAME, a catalogue entry) is itself a Jinja template compiled in, put in place of the file's: one path renders it, and what reads the template's text sees it.
  • As llama.cpp does, a leading BOS text produced by the template is removed when the tokenizer adds BOS itself.

Verification

tests/model/test_jinja.c renders, byte for byte like jinja2 in Hugging Face's environment (tools/jinja_ref.py): 157 committed cases covering the language and its error paths, and the published GLM-4.5 / 4.6 and GLM-4.7 templates (copied from llama.cpp's models/templates/ into tests/fixtures/jinja/) on a conversation with tools, calls, results and reasoning and on a lone question, thinking left to the template, on, off, and kept (clear_thinking: false); in make test-public the published templates of 11 families (SmolLM2, Qwen2.5, Qwen3, Llama 3.2, TinyLlama, Mistral, Phi-3.5, DeepSeek-R1, Gemma 3, Granite 3.3, Hermes 3) on six conversations each, with and without the generation prompt.

Consequences

Any model whose template stays within the supported language renders exactly as with transformers, without a per-model table. Recursive loops, loop.cycle() and call blocks are supported; template inheritance, include and import are rejected (they need a template loader, which chat templates never have) and such a template falls back to the built-in formats. Case conversions follow Python's str methods over all of Unicode, including the final-sigma rule; jinja2's title filter (which is not str.title()) is reproduced as such.