OKLMdocumentation v1.0.3
On this page

ADR-0018: The HTTPS API

Records what oklm serve exposes, who may call it, and how it shares one model between concurrent requests.

Status: accepted (2026-11-01).

Context

M8 puts a model behind an HTTP API so that any OpenAI-compatible client, the web dashboard and the native GUI can use it. The server runs on the user's machine, usually for the user alone, sometimes for a local network. A model answering over the network is a service others can reach: who may call it, and what it may do on the machine, must be decided before the first byte is served. OKLM depends only on the C library and nilorea: nilorea's HTTP/1.1 server (TLS, deadlines, limits, keep-alive) carries the transport, and streamed bodies were added to it for token streaming.

Decision

  • Program and library. oklm serve -m <model> in the oklm binary, over a library module (include/oklm/server/, src/server/) that the GUI can embed. One model per server.
  • Address and access.
    • Default address 127.0.0.1, port 7430.
    • On a loopback address, no key is needed.
    • On any other address, the server refuses to start unless it has a TLS certificate and key (--tls-cert, --tls-key) and at least one API key (--api-keys FILE, one key per line, a file only its owner can read). --insecure lifts both requirements, and the server says so at start and on the dashboard.
    • Keys come as Authorization: Bearer <key> and are compared in constant time. A missing or wrong key gets 401, without saying which.
  • Endpoints (OpenAI's shapes, so existing clients work unchanged):
    • POST /v1/chat/completions: messages (rendered with the model's chat template), max_tokens / max_completion_tokens, temperature, top_p, top_k, min_p, seed, stop, the penalties; tools and tool_choice (calls constrained by the tool-call grammar and returned as tool_calls); response_format json_object or json_schema (the output constrained by a grammar); stream (server-sent events, one chat.completion.chunk per piece, then data: [DONE]); usage in every final answer.
    • POST /v1/completions: a raw prompt, the same sampling and stream.
    • GET /v1/models: the loaded model.
    • GET /health: no key needed, for supervisors; says only whether the server is up.
    • GET /: the dashboard, one page embedded in the binary (no external assets): the model, the requests running and waiting, tokens per second, and a chat box that uses the API with the page's key. GET /api/stats gives the same figures as JSON.
  • Not exposed. The agent harness (its tools read, write and run commands on the server's machine, so offering them to a network needs its own decision); embeddings (no embedding models yet); images and audio (M7).
  • Sharing the model.
    • One context with --parallel sequences (default 4), each with the full context length.
    • A scheduler thread runs one forward pass per step over every active sequence (oklm_eval_batch()): new requests join between steps, and long prompts are cut into pieces of at most n_batch tokens so they do not stall the others' generation.
    • A request takes the free sequence holding the longest prefix of its prompt and keeps that prefix, so a conversation sent again re-evaluates only its new part.
    • Up to --queue requests (default 16) wait for a sequence; beyond that, 503 with Retry-After.
    • A client that disconnects (a failed stream write) frees its sequence at the next step.
  • Limits. A request body of at most 4 MiB. max_tokens is capped by what the sequence has room for. nilorea's deadlines apply to the head and the body, and to each piece of a stream.
  • Logs. One line per request: time, peer, endpoint, status, tokens in and out, duration. Prompts and outputs are not logged.

Consequences

  • Clients written for OpenAI's API work against a local model, and so do the dashboard and the GUI. They share one scheduler, so their requests batch together.
  • A server bound beyond loopback is encrypted and needs a key by default. The weaker setup has to be asked for by name.
  • Batched decoding gives a sequence what decoding it alone gives (oklm_eval_batch() matches separate evaluations), so a request's answer does not depend on what else runs (greedy, or with a fixed seed).
  • The harness stays off the network until a later ADR decides how a remote caller's actions are confined and approved.