ADR-0018: The HTTPS API
Records what oklm serve exposes, who may call it, and
how it shares one model between concurrent requests.
Status: accepted (2026-11-01).
Context
M8 puts a model behind an HTTP API so that any OpenAI-compatible client, the web dashboard and the native GUI can use it. The server runs on the user's machine, usually for the user alone, sometimes for a local network. A model answering over the network is a service others can reach: who may call it, and what it may do on the machine, must be decided before the first byte is served. OKLM depends only on the C library and nilorea: nilorea's HTTP/1.1 server (TLS, deadlines, limits, keep-alive) carries the transport, and streamed bodies were added to it for token streaming.
Decision
- Program and library.
oklm serve -m <model>in theoklmbinary, over a library module (include/oklm/server/,src/server/) that the GUI can embed. One model per server. - Address and access.
- Default address
127.0.0.1, port 7430. - On a loopback address, no key is needed.
- On any other address, the server refuses to start unless it has a
TLS certificate and key (
--tls-cert,--tls-key) and at least one API key (--api-keys FILE, one key per line, a file only its owner can read).--insecurelifts both requirements, and the server says so at start and on the dashboard. - Keys come as
Authorization: Bearer <key>and are compared in constant time. A missing or wrong key gets 401, without saying which.
- Default address
- Endpoints (OpenAI's shapes, so existing clients
work unchanged):
POST /v1/chat/completions:messages(rendered with the model's chat template),max_tokens/max_completion_tokens,temperature,top_p,top_k,min_p,seed,stop, the penalties;toolsandtool_choice(calls constrained by the tool-call grammar and returned astool_calls);response_formatjson_objectorjson_schema(the output constrained by a grammar);stream(server-sent events, onechat.completion.chunkper piece, thendata: [DONE]);usagein every final answer.POST /v1/completions: a rawprompt, the same sampling andstream.GET /v1/models: the loaded model.GET /health: no key needed, for supervisors; says only whether the server is up.GET /: the dashboard, one page embedded in the binary (no external assets): the model, the requests running and waiting, tokens per second, and a chat box that uses the API with the page's key.GET /api/statsgives the same figures as JSON.
- Not exposed. The agent harness (its tools read, write and run commands on the server's machine, so offering them to a network needs its own decision); embeddings (no embedding models yet); images and audio (M7).
- Sharing the model.
- One context with
--parallelsequences (default 4), each with the full context length. - A scheduler thread runs one forward pass per step over every active
sequence (
oklm_eval_batch()): new requests join between steps, and long prompts are cut into pieces of at mostn_batchtokens so they do not stall the others' generation. - A request takes the free sequence holding the longest prefix of its prompt and keeps that prefix, so a conversation sent again re-evaluates only its new part.
- Up to
--queuerequests (default 16) wait for a sequence; beyond that, 503 withRetry-After. - A client that disconnects (a failed stream write) frees its sequence at the next step.
- One context with
- Limits. A request body of at most 4 MiB.
max_tokensis capped by what the sequence has room for. nilorea's deadlines apply to the head and the body, and to each piece of a stream. - Logs. One line per request: time, peer, endpoint, status, tokens in and out, duration. Prompts and outputs are not logged.
Consequences
- Clients written for OpenAI's API work against a local model, and so do the dashboard and the GUI. They share one scheduler, so their requests batch together.
- A server bound beyond loopback is encrypted and needs a key by default. The weaker setup has to be asked for by name.
- Batched decoding gives a sequence what decoding it alone gives
(
oklm_eval_batch()matches separate evaluations), so a request's answer does not depend on what else runs (greedy, or with a fixed seed). - The harness stays off the network until a later ADR decides how a remote caller's actions are confined and approved.