OKLMdocumentation v1.0.3
On this page

Context, compaction and sessions

How oklm agent keeps a conversation within its context (accounting, old tool results cleared, summaries) and on disk (transcripts, resuming, rewinding). Part of the agent harness (harness.md).

Status: implemented.

Context and compaction

src/harness/compact.c, on the summary and the threshold rule the chats share (src/client/compaction.c, inference.md). Each message's tokens are counted once (the model's tokenizer, or 4 bytes a token with a backend), so are the system prompt, its memory files and the tool definitions; after each turn the sum is scaled by the model's own count of the prompt, which covers the template's markers. oklm_agent_context() gives the figures, /context shows them as a bar and per part, with the threshold.

Before each model turn, room for the turn is looked at first. Then, when the prompt passes the threshold (compactThreshold × (window − reply reserve), the reserve being the turn's token limit, at most a quarter of the window; without a setting the share follows the model's size: 0.60 under 2 billion parameters, 0.75 under 8 billion, 0.85 above, and for a model behind --url):

  1. Old tool results are cleared: those longer than 200 tokens, older than the last two turns with calls, read [output cleared — re-run if needed]. When that is enough, nothing else happens.
  2. Otherwise the conversation is summarized by the model, when the summary frees room (below): the summary request (prompts/compact.md: the user's requests, decisions, files, errors and fixes, the state, open tasks, the next step) follows the conversation, so the in-process model's cache is reused. When the conversation and the request do not fit, the summary is made from a transcript read in parts (prompts/compact-system.md), each part's summary carried into the next.
  3. The history becomes: the summary with the user's messages verbatim (kept by the agent through every compaction; the first one and the latest ones when they do not all fit an eighth of the window); the user's latest message as it was; the task list, the files changed lately as they are now (an eighth of the window; recorded as read, so they can be edited at once), the sub-directories' instructions (oklm_memory_loaded_notes()); and, mid-task, the turn with calls in progress and its results (cut to a quarter of the window).

A summary that would free nothing is not made. The note costs tokens of its own: the summary (an eighth of the window, 512 to 4096 tokens), its first sentence, the user's messages given back verbatim. The messages it would replace (all but the user's latest and the exchange in progress) must hold an eighth of the window more than the summary and its sentence may take, which is the rule of the chats (oklm_compact_gains()); else the prompt goes as it is. So a prompt that is past the threshold from the first message, because the system prompt and the tools alone take that much of a small window, is not compacted at every turn into a longer one. The rule holds for every compaction the agent decides by itself (the threshold, a prompt the model refuses, a full context that cut an answer); /compact is the person's and is always made.

A prompt the model finds too long (OKLM_ERR_OVERFLOW: the in-process context does not hold it, or an oklm serve answers the prompt holds N tokens, the context M) is given a larger context where one can be made (below), else compacted when that frees room, and the turn tried again; a summary is not made twice without a model turn between. /compact [focus] (oklm_agent_compact()) does it on request, the focus added to the request; Esc or Ctrl-C interrupts it and leaves the conversation as it was. Compactions come as OKLM_AGENT_EV_COMPACT events, which the terminal shows; after a turn past 70% of the window it prints a warning. /clear (or /reset) starts a new conversation.

Room for a turn

The agent's system prompt and its tools are in every prompt: about 4600 tokens with the default tools, as Qwen3-0.6B counts them. A window they nearly fill holds no conversation, and no summary changes that. So, once the model has counted a prompt (the agent's own count before that may be far from the model's: a template may write the tools shorter than their JSON, and a server's tokenizer is not the agent's), the agent holds two things against the window before each turn, and when the model refuses a prompt for its length:

  • The conversation with a turn's room. A turn needs 512 tokens beyond the prompt, an eighth of the window when the window is under 4096 tokens, and no more than the turn's own token limit (--max-tokens 16 needs 16). When the prompt leaves less, a larger context is asked for first, by the same two ways as for a cut answer (in process when the memory plan holds it, on an oklm serve that was not started with --fixed-context after its dry run), as often as it takes: (the conversation no longer fitted the context, 4096 tokens: it is now 8192 tokens). When none is made, the threshold's rule above applies: the conversation is summarized when that frees room.

  • The agent's own part. When the system prompt and the tools alone leave less than a turn's room, and the context cannot be made larger, no request is sent (the server would refuse it, or cut the answer after a few tokens). The message is not answered, and the agent says why, with the figures and the ways out:

    (no room for a conversation in the context: the agent's system prompt and tools take 4653 tokens (about 1855 and 2797) of 4692, and a turn needs 512 more. A larger context holds them: oklm serve -c N where the server is started, /context-size N on a server that resizes its own; the settings' contextWindow tells the size of a server that does not tell it. Fewer tools take less: --tool-rag K sends the K a message needs.)

    oklm_agent_send() returns OKLM_ERR_OVERFLOW with that text as the reply, and oklm_agent_no_room() gives it until the next message; oklm_agent_context() tells the two figures (fixed, room_min). At a prompt the person makes room (/context-size N) and goes on; oklm agent -p says it on stderr and ends with status 1: the message was not answered. So does a message the model or the server refused for its length once a larger context and a summary were tried ((the conversation no longer fits the context; /compact or /clear)). The JSON outputs tell it as any failed request ("is_error":true, the text as the result), status 1 too. The status follows a refusal that happened, never the agent's own count; an answer that was written, even cut by --max-tokens, is status 0. (Up to 1.0.0 the text output ended with status 0 in these two cases.)

Measured with Qwen3-0.6B on the CPU and oklm agent --url ... -p "Say hello in one word.":

Server Before Now
oklm serve --fixed-context -c 4692 two compactions, each prompt longer (4221 to 4346 tokens by the agent's count; 4676 to the server), then 400: the prompt holds 4949 tokens, the context 4692, and the program aborted (a double free, below) one request (4666 tokens, the answer cut after 26); no summary; the message above; the next message is refused without a request
oklm serve --fixed-context -c 4400 the server refuses the prompt (the prompt holds 4666 tokens, the context 4400); the message above with of 4400
oklm serve -c 4692 the answer the context cut is gone on with in a context of 9384 tokens (POST /api/context), and ends: hello
in process, -m ... -c 4096 (the conversation no longer fitted the context, 4096 tokens: it is now 8192 tokens), and the turn is answered

What stays as it was: before any prompt was counted, a first request may be sent to a window that turns out too small (the first two rows: one request each); a server other than oklm serve that refuses a long prompt is an error of the server's, as before.

An answer with reasoning that the context cut, on a model that writes reasoning, was summarized before going on by a turn that freed the answer's reasoning under it: the program ended on a double free. The summary's turn now sets the reasoning of the turn in progress aside and gives it back (agent_ask_private()).

tests/harness/test_agent.c (test_room, on the scripted backend): a window with a turn's room and no more, past the threshold from the first message, is asked no summary and answers; a backend that refuses the prompt for its length gets the message above, no summary, and the next message is refused with no request, or, with a program that makes the window larger, the larger window and the answer; an answer cut after a few tokens in such a window stays cut, with the message; a conversation that no longer fits before a turn is given the larger window, or summarized where the window stays; an answer with reasoning is summarized and gone on with.

tests/harness/test_compact.c: the figures of /context and the threshold; /compact with a focus (the history becomes three messages, the file given back editable without a Read), interrupted (nothing changed); a summary from a transcript in parts, each prompt within the window; and the acceptance, a synthetic session of 40 requests and 440 turns in a window that holds about four steps, by a scripted developer that reads its progress from the prompt alone: no prompt passes the window, 60 summaries and 85 clearings later the first request and a rule given at the tenth are still there verbatim, and the log holds the 200 steps once each, in order. With Qwen3-4B and a 12288-token context, eight files read and logged one at a time: old results cleared (8847 tokens to 4884), then a summary mid-task (8393 to 2142), after which the model went on with the fourth file and finished the log correctly.

A cut answer

An answer the turn's token limit or the context cut is gone on with (since 2026-11-16), by the rule oklm chat and the chat window share (inference.md, src/client/reply_continue.c): the conversation is given to the model again with the answer so far as the open start of its turn, and what the model adds is appended to the same message.

  • In process the answer's text follows the rendered prompt, tokenized apart from it, so the rows are those the cache holds.

  • A backend gets the answer as a last assistant message with "prefix": true; oklm serve continues it (server.md). Another server may answer it as a new turn. Its reasoning so far goes back the way it came: as reasoning_content when the server gave it apart, and inside the content, as the server wrote it, when the content began with the model's <think> block (closed, empty, or still open when the answer was cut inside it), so the model goes on from the text it wrote.

  • A model that writes its reasoning in a channel (harmony, Gemma 4, Muse Glimmer) and is cut inside the first header of its turn has written markup alone: it is dropped and the turn is asked afresh (server.md).

  • The context. When the context is what cut the answer, room is made first, by the rule of the chats (inference.md): a larger context when one can be had, else a compaction, and a notice says which. The library asks the program that owns the context (oklm_agent_params.grow_context: oklm agent gives it the two ways below; a program that gives none gets the compaction), and oklm_agent_set_window() tells the agent a server's size.

    • In process, as oklm chat grows its own: twice the size, the model's trained context at most, when the memory plan holds it. The conversation is evaluated again in the new context, and the notice is (the context was full, 4701 tokens: it is now 9402 tokens, and the answer goes on).
    • On a server (--url, an oklm serve that was not started with --fixed-context): the same size is asked of the server (POST /api/context), when its own plan for it (the request's dry_run) says it fits its memory, holds a conversation that fills it and takes no more than half of the free memory, which is how the chat window judges a size (gui.md). The agent then counts against the new size.
    • Otherwise (the plan refuses, the trained context is reached, the server is --fixed-context or is not oklm serve, a background agent is running): the conversation is compacted, (the context was full: the conversation was summarized, and the answer goes on), when autoCompact is on and the summary frees room (above); with it off the full context is said, with /compact, /continue and /context-size N. When no summary would free room the answer stays cut, and the notice says which of the two fills the context: the answer, or the agent's own system prompt and tools (Room for a turn).

    A larger context is tried whatever autoCompact says, as in the chats; /context-size N keeps its meaning (the size the person chooses, force included). With --url and no contextWindow in the settings, the window the agent counts against is the one the server tells (GET /api/context: its n_ctx), 32768 for a server that tells none.

  • The token limit. The 8192 tokens a turn has by default are the size of one piece. --max-tokens N is the user's: an answer it cuts ends there.

  • The bound. 8 pieces in a row at most (continueMax, --continue-max N, /autocontinue N); an answer that repeats itself is not continued. A notice says so and that /continue goes on.

  • Tool calls. A piece that ends in a tool call is the answer's: the call is run and the work goes on. An answer cut inside a tool call is continued too: the half that was written is not parsed nor run, the next piece goes on under the calls' grammar from the call's text so far (in process), or a server is sent that text as tool_call_open and completes it (server.md), and the call that comes out whole is run once. With continuing manual the cut is said and nothing runs until /continue ends the call. A backend says a turn was cut inside a call with oklm_agent_turn.call_open.

  • Manual. autoContinue false, --auto-continue off or /autocontinue off: a cut answer says (The reply was cut by the reply's token limit: /continue goes on with it.), and /continue (oklm_agent_continue()) goes on, past a limit the user gave too.

test_continue_grow (the scripted backend, a window of 100000 tokens told full): a program that makes the context larger is asked once, the answer goes on with no summary asked and the window is 200000 from then on, compaction manual or not; one that cannot leaves the conversation to be summarized. test_agent_grow (tests/harness/test_cli_chat.c): oklm agent in process on Qwen3-0.6B Q8_0, the CPU, greedy, with a prompt of 4637 tokens in a context of 4701: the context is made 9402 and the answer, 1468 bytes, is the one written in a context of 16384. The same question through oklm serve -c 4692 (a prompt of 4628 tokens): the server's context is made 9384 by its route and the answer is the one of a server started with -c 16384.

tests/harness/test_agent.c (test_continue_answer), on the scripted backend: an answer in two turns of the model is one message, the second request's last message being the first turn's text with "prefix": true; manual, the cut is said and oklm_agent_continue() ends it; four turns that all stop at the limit are three pieces with a bound of 2, then the notice; a piece that calls Read has the call run; a turn cut inside a call to Read, twice, sends the call's growing text back as tool_call_open and runs the call once when it comes whole, by itself or at the second /continue; with a window of 100000 tokens told full, the conversation is summarized and the answer goes on from the summary, or, compaction manual, the full context is said. tests/harness/test_cli_chat.c (test_agent_continue) types /continue, /autocontinue and /autocompact into oklm agent --mock. With Qwen3-0.6B in process on the CPU and --max-tokens 40, an answer cut inside its reasoning went on reasoning at the first /continue and closed its </think> and began its answer at the second. With the same model in process and turns written in pieces of 6 tokens (test_continue_call_model), a request to read a file has its call to Read cut inside several times and completed: Read {"file_path":"p1.txt"}, the call and the arguments of the run whose turns are written in one go, run once. The answer after the tool's result is not the same text in the two runs (it differs by two backquotes around the file's name). The agent and oklm serve together on that model (test_agent_call_joint in tests/server/test_server.c, the CPU): with pieces of 6 tokens the server answers each cut with the call's text so far and the agent sends it back, the call that comes out is Read {"file_path":"p1.txt"} as in the run in one go, run once, and the answer after the tool's result, whose reasoning is cut several times too, is the same text in the two runs.

Sessions

src/harness/session.c. Each run of oklm agent is a session; its id (the date and time and a random suffix) also names the audit log and the jobs' journal. The conversation is written to .oklm/sessions/<id>.jsonl as it goes, one JSON object per line, appended, flushed and synced as each is written: the request before the model's turn, each answer once its counts are known, each tool result as it comes. A line cut short by a crash is skipped on reading, and the next line starts on a line of its own. Messages carry their id and their parent's: the conversation is the path from the last message back to the first. A compaction starts a new path, with a line linking it to the old one (the user's messages from before it are still found for the next compaction); /clear and a rewind write where the conversation now ends; nothing written is rewritten.

  • --resume resumes the project's most recent session, --resume ID that one; /resume in a session lists them (date, messages, the first line of the first request) and reads a number. The conversation comes back whole, ids and times included; the user's messages, and the files edited, come back for the next compaction. A call that was running when the process ended gets a result saying it was interrupted, so the history stays whole for a backend. The same session goes on: its transcript, its audit log, its jobs (re-attached or found ended). --fork-session copies the conversation into a new session instead. -c stays the context's length: --resume has no short form. It is one word in oklm agent and oklm chat: --continue, which it was before /continue came to mean going on with a cut answer, is refused with a line that names --resume.
  • Warm resume (--kv-save, off by default; an in-process model): the model's cache is kept in .oklm/sessions/<id>.kv beside the transcript (oklm_agent_set_kv_save(), a KV file): after each model turn the rows of its new tokens (the prompt's and the reply's) are appended. A resumed session's first turn, on a context holding nothing, loads the rows of the file that match its rendered prompt and evaluates only the tokens after them, so a long conversation does not prefill again. The file follows the session as /branch, /resume, /fork and /background switch it; a file of another model or cache type, or of a conversation that no longer matches (a compaction, a /clear), loads nothing or only the common prefix, and the next save rewrites it. Asides (titles, /btw, summaries) neither save nor load. The file holds the whole cache: about 128 KB per token for an 8B model with an F16 cache.
  • Asides and the cache. Without a small model, the asides with the conversation (a title, /btw, a recap) run on the main context after the conversation's rows, so the next turn still finds the conversation cached (only the aside's question and answer are dropped); with a smallModel, they run on its own context and the main cache is not touched. The rolling summaries of a compaction start from their own prompt on the main context, but the compaction replaces the conversation anyway. test_agent_resume in tests/server/test_server.c: a title asked between two turns, then the session resumed in a new context, which finds the conversation cached and answers as the agent that never stopped.
  • Before Write or Edit first changes a file during a turn (a user message and what follows it), its content is copied to .oklm/sessions/<id>/<n> (up to 16 MiB; a file that did not exist is noted as such), sub-agents' edits included.
  • /rewind, or Esc Esc on an empty line, lists the user's messages with the files changed since each; picked one, it asks what goes back to before it: the conversation, the files (those created removed), or both. Commands' effects are not tracked, and it says so. The message comes back at the prompt to send again or change. A rewound conversation goes on as a new branch.
  • /export [FILE] writes the conversation as Markdown (the requests, the answers, the calls, the start of each result; the agent's reminders left out): to the file, else to the clipboard (wl-copy, xclip, xsel, pbcopy, clip), else to conversation-<id>.md.

Sessions without an agent

A project's sessions are listed, read and removed without a model and without an agent, by oklm agent sessions and by the sessions browser of oklm-gui (gui.md):

oklm agent sessions [--project DIR]                 # id, date, turns, model, title: the most recent first
oklm agent sessions --json                          # an array: id, title, model, created, time, turns, messages, bytes
oklm agent sessions export ID                       # Markdown on standard output
oklm agent sessions export ID --format json -o FILE # one JSON object, every message whole
oklm agent sessions rm ID                           # the transcript, its file copies and git references
  • Markdown is /export's form under the session's title, with a line naming the session, its model and its start.
  • JSON is {"id", "title", "project", "model", "created", "messages": [...]}. A message holds role, content and time (milliseconds since the epoch), and what its transcript line has: meta (true for a reminder of the agent's), name and tool_call_id for a tool's result, reasoning_content, calls ([{"id", "name", "arguments"}], the arguments as a JSON string), prompt_tokens and output_tokens. The messages are the conversation the transcript ends with, those from before a compaction first.
  • Exit status 1 when the session is not there or the file cannot be written, 2 for a usage error.

In oklm/harness/session.h: oklm_session_open_read() (nothing is written, so a session an agent is running can be read), oklm_session_messages(), oklm_session_search() (a text in the messages and the calls, whatever the case of its ASCII letters: a hit a line), oklm_session_export(), oklm_session_remove(), and the model, start, size and turns in oklm_session_list()'s entries. tests/harness/test_session.c (test_read_only) and tests/harness/test_cli_sessions.c (the command, on a session made by a scripted agent).

  • At start, sessions not written to for cleanupPeriodDays (30) are removed with their copies; 0 turns the transcripts off.

tests/harness/test_session.c: the conversation read back is the same, message by message, and the work goes on from it; a line cut short; a transcript ending with a call and no result; a process killed (SIGKILL) between a Read's result and the Write after it, then resumed in another process, which finishes the task; the files restored by rewind (to before any message, those created removed), the conversation rewound into a branch the transcript then ends with, and both after resuming; compactions and clears; a fork; the list, the latest, the cleanup; the Markdown export.