Context, compaction and sessions
How oklm agent keeps a conversation within its context
(accounting, old tool results cleared, summaries) and on disk
(transcripts, resuming, rewinding). Part of the agent harness (harness.md).
Status: implemented.
Context and compaction
src/harness/compact.c, on the summary and the threshold
rule the chats share (src/client/compaction.c, inference.md). Each message's
tokens are counted once (the model's tokenizer, or 4 bytes a token with
a backend), so are the system prompt, its memory files and the tool
definitions; after each turn the sum is scaled by the model's own count
of the prompt, which covers the template's markers.
oklm_agent_context() gives the figures,
/context shows them as a bar and per part, with the
threshold.
Before each model turn, room for the
turn is looked at first. Then, when the prompt passes the threshold
(compactThreshold × (window − reply reserve), the reserve
being the turn's token limit, at most a quarter of the window; without a
setting the share follows the model's size: 0.60 under 2 billion
parameters, 0.75 under 8 billion, 0.85 above, and for a model behind
--url):
- Old tool results are cleared: those longer than 200
tokens, older than the last two turns with calls, read
[output cleared — re-run if needed]. When that is enough, nothing else happens. - Otherwise the conversation is summarized by the
model, when the summary frees room (below): the summary request
(
prompts/compact.md: the user's requests, decisions, files, errors and fixes, the state, open tasks, the next step) follows the conversation, so the in-process model's cache is reused. When the conversation and the request do not fit, the summary is made from a transcript read in parts (prompts/compact-system.md), each part's summary carried into the next. - The history becomes: the summary with the user's
messages verbatim (kept by the agent through every compaction; the first
one and the latest ones when they do not all fit an eighth of the
window); the user's latest message as it was; the task list, the files
changed lately as they are now (an eighth of the window; recorded as
read, so they can be edited at once), the sub-directories' instructions
(
oklm_memory_loaded_notes()); and, mid-task, the turn with calls in progress and its results (cut to a quarter of the window).
A summary that would free nothing is not made. The
note costs tokens of its own: the summary (an eighth of the window, 512
to 4096 tokens), its first sentence, the user's messages given back
verbatim. The messages it would replace (all but the user's latest and
the exchange in progress) must hold an eighth of the window more than
the summary and its sentence may take, which is the rule of the chats
(oklm_compact_gains()); else the prompt goes as it is. So a
prompt that is past the threshold from the first message, because the
system prompt and the tools alone take that much of a small window, is
not compacted at every turn into a longer one. The rule holds for every
compaction the agent decides by itself (the threshold, a prompt the
model refuses, a full context that cut an answer); /compact
is the person's and is always made.
A prompt the model finds too long (OKLM_ERR_OVERFLOW:
the in-process context does not hold it, or an oklm serve
answers the prompt holds N tokens, the context M) is given
a larger context where one can be made (below), else compacted when that frees room,
and the turn tried again; a summary is not made twice without a model
turn between. /compact [focus]
(oklm_agent_compact()) does it on request, the focus added
to the request; Esc or Ctrl-C interrupts it and leaves the conversation
as it was. Compactions come as OKLM_AGENT_EV_COMPACT
events, which the terminal shows; after a turn past 70% of the window it
prints a warning. /clear (or /reset) starts a
new conversation.
Room for a turn
The agent's system prompt and its tools are in every prompt: about 4600 tokens with the default tools, as Qwen3-0.6B counts them. A window they nearly fill holds no conversation, and no summary changes that. So, once the model has counted a prompt (the agent's own count before that may be far from the model's: a template may write the tools shorter than their JSON, and a server's tokenizer is not the agent's), the agent holds two things against the window before each turn, and when the model refuses a prompt for its length:
The conversation with a turn's room. A turn needs 512 tokens beyond the prompt, an eighth of the window when the window is under 4096 tokens, and no more than the turn's own token limit (
--max-tokens 16needs 16). When the prompt leaves less, a larger context is asked for first, by the same two ways as for a cut answer (in process when the memory plan holds it, on anoklm servethat was not started with--fixed-contextafter its dry run), as often as it takes:(the conversation no longer fitted the context, 4096 tokens: it is now 8192 tokens). When none is made, the threshold's rule above applies: the conversation is summarized when that frees room.The agent's own part. When the system prompt and the tools alone leave less than a turn's room, and the context cannot be made larger, no request is sent (the server would refuse it, or cut the answer after a few tokens). The message is not answered, and the agent says why, with the figures and the ways out:
(no room for a conversation in the context: the agent's system prompt and tools take 4653 tokens (about 1855 and 2797) of 4692, and a turn needs 512 more. A larger context holds them: oklm serve -c N where the server is started, /context-size N on a server that resizes its own; the settings' contextWindow tells the size of a server that does not tell it. Fewer tools take less: --tool-rag K sends the K a message needs.)oklm_agent_send()returnsOKLM_ERR_OVERFLOWwith that text as the reply, andoklm_agent_no_room()gives it until the next message;oklm_agent_context()tells the two figures (fixed,room_min). At a prompt the person makes room (/context-size N) and goes on;oklm agent -psays it on stderr and ends with status 1: the message was not answered. So does a message the model or the server refused for its length once a larger context and a summary were tried ((the conversation no longer fits the context; /compact or /clear)). The JSON outputs tell it as any failed request ("is_error":true, the text as theresult), status 1 too. The status follows a refusal that happened, never the agent's own count; an answer that was written, even cut by--max-tokens, is status 0. (Up to 1.0.0 the text output ended with status 0 in these two cases.)
Measured with Qwen3-0.6B on the CPU and
oklm agent --url ... -p "Say hello in one word.":
| Server | Before | Now |
|---|---|---|
oklm serve --fixed-context -c 4692 |
two compactions, each prompt longer (4221 to 4346 tokens by the
agent's count; 4676 to the server), then
400: the prompt holds 4949 tokens, the context 4692, and
the program aborted (a double free, below) |
one request (4666 tokens, the answer cut after 26); no summary; the message above; the next message is refused without a request |
oklm serve --fixed-context -c 4400 |
the server refuses the prompt
(the prompt holds 4666 tokens, the context 4400); the
message above with of 4400 |
|
oklm serve -c 4692 |
the answer the context cut is gone on with in a context of 9384
tokens (POST /api/context), and ends:
hello |
|
in process, -m ... -c 4096 |
(the conversation no longer fitted the context, 4096 tokens: it is now 8192 tokens),
and the turn is answered |
What stays as it was: before any prompt was counted, a first request
may be sent to a window that turns out too small (the first two rows:
one request each); a server other than oklm serve that
refuses a long prompt is an error of the server's, as before.
An answer with reasoning that the context cut, on a
model that writes reasoning, was summarized before going on by a turn
that freed the answer's reasoning under it: the program ended on a
double free. The summary's turn now sets the reasoning of the turn in
progress aside and gives it back (agent_ask_private()).
tests/harness/test_agent.c (test_room, on
the scripted backend): a window with a turn's room and no more, past the
threshold from the first message, is asked no summary and answers; a
backend that refuses the prompt for its length gets the message above,
no summary, and the next message is refused with no request, or, with a
program that makes the window larger, the larger window and the answer;
an answer cut after a few tokens in such a window stays cut, with the
message; a conversation that no longer fits before a turn is given the
larger window, or summarized where the window stays; an answer with
reasoning is summarized and gone on with.
tests/harness/test_compact.c: the figures of
/context and the threshold; /compact with a
focus (the history becomes three messages, the file given back editable
without a Read), interrupted (nothing changed); a summary from a
transcript in parts, each prompt within the window; and the acceptance,
a synthetic session of 40 requests and 440 turns in a window that holds
about four steps, by a scripted developer that reads its progress from
the prompt alone: no prompt passes the window, 60 summaries and 85
clearings later the first request and a rule given at the tenth are
still there verbatim, and the log holds the 200 steps once each, in
order. With Qwen3-4B and a 12288-token context, eight files read and
logged one at a time: old results cleared (8847 tokens to 4884), then a
summary mid-task (8393 to 2142), after which the model went on with the
fourth file and finished the log correctly.
A cut answer
An answer the turn's token limit or the context cut is gone on with
(since 2026-11-16), by the rule oklm chat and the chat
window share (inference.md,
src/client/reply_continue.c): the conversation is given to
the model again with the answer so far as the open start of its turn,
and what the model adds is appended to the same message.
In process the answer's text follows the rendered prompt, tokenized apart from it, so the rows are those the cache holds.
A backend gets the answer as a last
assistantmessage with"prefix": true;oklm servecontinues it (server.md). Another server may answer it as a new turn. Its reasoning so far goes back the way it came: asreasoning_contentwhen the server gave it apart, and inside thecontent, as the server wrote it, when thecontentbegan with the model's<think>block (closed, empty, or still open when the answer was cut inside it), so the model goes on from the text it wrote.A model that writes its reasoning in a channel (harmony, Gemma 4, Muse Glimmer) and is cut inside the first header of its turn has written markup alone: it is dropped and the turn is asked afresh (server.md).
The context. When the context is what cut the answer, room is made first, by the rule of the chats (inference.md): a larger context when one can be had, else a compaction, and a notice says which. The library asks the program that owns the context (
oklm_agent_params.grow_context:oklm agentgives it the two ways below; a program that gives none gets the compaction), andoklm_agent_set_window()tells the agent a server's size.- In process, as
oklm chatgrows its own: twice the size, the model's trained context at most, when the memory plan holds it. The conversation is evaluated again in the new context, and the notice is(the context was full, 4701 tokens: it is now 9402 tokens, and the answer goes on). - On a server (
--url, anoklm servethat was not started with--fixed-context): the same size is asked of the server (POST /api/context), when its own plan for it (the request'sdry_run) says it fits its memory, holds a conversation that fills it and takes no more than half of the free memory, which is how the chat window judges a size (gui.md). The agent then counts against the new size. - Otherwise (the plan refuses, the trained context is
reached, the server is
--fixed-contextor is notoklm serve, a background agent is running): the conversation is compacted,(the context was full: the conversation was summarized, and the answer goes on), whenautoCompactis on and the summary frees room (above); with it off the full context is said, with/compact,/continueand/context-size N. When no summary would free room the answer stays cut, and the notice says which of the two fills the context: the answer, or the agent's own system prompt and tools (Room for a turn).
A larger context is tried whatever
autoCompactsays, as in the chats;/context-size Nkeeps its meaning (the size the person chooses,forceincluded). With--urland nocontextWindowin the settings, the window the agent counts against is the one the server tells (GET /api/context: itsn_ctx), 32768 for a server that tells none.- In process, as
The token limit. The 8192 tokens a turn has by default are the size of one piece.
--max-tokens Nis the user's: an answer it cuts ends there.The bound. 8 pieces in a row at most (
continueMax,--continue-max N,/autocontinue N); an answer that repeats itself is not continued. A notice says so and that/continuegoes on.Tool calls. A piece that ends in a tool call is the answer's: the call is run and the work goes on. An answer cut inside a tool call is continued too: the half that was written is not parsed nor run, the next piece goes on under the calls' grammar from the call's text so far (in process), or a server is sent that text as
tool_call_openand completes it (server.md), and the call that comes out whole is run once. With continuing manual the cut is said and nothing runs until/continueends the call. A backend says a turn was cut inside a call withoklm_agent_turn.call_open.Manual.
autoContinuefalse,--auto-continue offor/autocontinue off: a cut answer says(The reply was cut by the reply's token limit: /continue goes on with it.), and/continue(oklm_agent_continue()) goes on, past a limit the user gave too.
test_continue_grow (the scripted backend, a window of
100000 tokens told full): a program that makes the context larger is
asked once, the answer goes on with no summary asked and the window is
200000 from then on, compaction manual or not; one that cannot leaves
the conversation to be summarized. test_agent_grow
(tests/harness/test_cli_chat.c): oklm agent in
process on Qwen3-0.6B Q8_0, the CPU, greedy, with a prompt of 4637
tokens in a context of 4701: the context is made 9402 and the answer,
1468 bytes, is the one written in a context of 16384. The same question
through oklm serve -c 4692 (a prompt of 4628 tokens): the
server's context is made 9384 by its route and the answer is the one of
a server started with -c 16384.
tests/harness/test_agent.c
(test_continue_answer), on the scripted backend: an answer
in two turns of the model is one message, the second request's last
message being the first turn's text with "prefix": true;
manual, the cut is said and oklm_agent_continue() ends it;
four turns that all stop at the limit are three pieces with a bound of
2, then the notice; a piece that calls Read has the call
run; a turn cut inside a call to Read, twice, sends the
call's growing text back as tool_call_open and runs the
call once when it comes whole, by itself or at the second
/continue; with a window of 100000 tokens told full, the
conversation is summarized and the answer goes on from the summary, or,
compaction manual, the full context is said.
tests/harness/test_cli_chat.c
(test_agent_continue) types /continue,
/autocontinue and /autocompact into
oklm agent --mock. With Qwen3-0.6B in process on the CPU
and --max-tokens 40, an answer cut inside its reasoning
went on reasoning at the first /continue and closed its
</think> and began its answer at the second. With the
same model in process and turns written in pieces of 6 tokens
(test_continue_call_model), a request to read a file has
its call to Read cut inside several times and completed:
Read {"file_path":"p1.txt"}, the call and the arguments of
the run whose turns are written in one go, run once. The answer after
the tool's result is not the same text in the two runs (it differs by
two backquotes around the file's name). The agent and
oklm serve together on that model
(test_agent_call_joint in
tests/server/test_server.c, the CPU): with pieces of 6
tokens the server answers each cut with the call's text so far and the
agent sends it back, the call that comes out is
Read {"file_path":"p1.txt"} as in the run in one go, run
once, and the answer after the tool's result, whose reasoning is cut
several times too, is the same text in the two runs.
Sessions
src/harness/session.c. Each run of
oklm agent is a session; its id (the date and time and a
random suffix) also names the audit log and the jobs' journal. The
conversation is written to .oklm/sessions/<id>.jsonl
as it goes, one JSON object per line, appended, flushed and synced as
each is written: the request before the model's turn, each answer once
its counts are known, each tool result as it comes. A line cut short by
a crash is skipped on reading, and the next line starts on a line of its
own. Messages carry their id and their parent's: the conversation is the
path from the last message back to the first. A compaction starts a new
path, with a line linking it to the old one (the user's messages from
before it are still found for the next compaction); /clear
and a rewind write where the conversation now ends; nothing written is
rewritten.
--resumeresumes the project's most recent session,--resume IDthat one;/resumein a session lists them (date, messages, the first line of the first request) and reads a number. The conversation comes back whole, ids and times included; the user's messages, and the files edited, come back for the next compaction. A call that was running when the process ended gets a result saying it was interrupted, so the history stays whole for a backend. The same session goes on: its transcript, its audit log, its jobs (re-attached or found ended).--fork-sessioncopies the conversation into a new session instead.-cstays the context's length:--resumehas no short form. It is one word inoklm agentandoklm chat:--continue, which it was before/continuecame to mean going on with a cut answer, is refused with a line that names--resume.- Warm resume (
--kv-save, off by default; an in-process model): the model's cache is kept in.oklm/sessions/<id>.kvbeside the transcript (oklm_agent_set_kv_save(), a KV file): after each model turn the rows of its new tokens (the prompt's and the reply's) are appended. A resumed session's first turn, on a context holding nothing, loads the rows of the file that match its rendered prompt and evaluates only the tokens after them, so a long conversation does not prefill again. The file follows the session as/branch,/resume,/forkand/backgroundswitch it; a file of another model or cache type, or of a conversation that no longer matches (a compaction, a/clear), loads nothing or only the common prefix, and the next save rewrites it. Asides (titles,/btw, summaries) neither save nor load. The file holds the whole cache: about 128 KB per token for an 8B model with an F16 cache. - Asides and the cache. Without a small model, the
asides with the conversation (a title,
/btw, a recap) run on the main context after the conversation's rows, so the next turn still finds the conversation cached (only the aside's question and answer are dropped); with asmallModel, they run on its own context and the main cache is not touched. The rolling summaries of a compaction start from their own prompt on the main context, but the compaction replaces the conversation anyway.test_agent_resumeintests/server/test_server.c: a title asked between two turns, then the session resumed in a new context, which finds the conversation cached and answers as the agent that never stopped. - Before Write or Edit first changes a file during a turn (a user
message and what follows it), its content is copied to
.oklm/sessions/<id>/<n>(up to 16 MiB; a file that did not exist is noted as such), sub-agents' edits included. /rewind, or Esc Esc on an empty line, lists the user's messages with the files changed since each; picked one, it asks what goes back to before it: the conversation, the files (those created removed), or both. Commands' effects are not tracked, and it says so. The message comes back at the prompt to send again or change. A rewound conversation goes on as a new branch./export [FILE]writes the conversation as Markdown (the requests, the answers, the calls, the start of each result; the agent's reminders left out): to the file, else to the clipboard (wl-copy,xclip,xsel,pbcopy,clip), else toconversation-<id>.md.
Sessions without an agent
A project's sessions are listed, read and removed without a model and
without an agent, by oklm agent sessions and by the
sessions browser of oklm-gui (gui.md):
oklm agent sessions [--project DIR] # id, date, turns, model, title: the most recent first
oklm agent sessions --json # an array: id, title, model, created, time, turns, messages, bytes
oklm agent sessions export ID # Markdown on standard output
oklm agent sessions export ID --format json -o FILE # one JSON object, every message whole
oklm agent sessions rm ID # the transcript, its file copies and git references
- Markdown is
/export's form under the session's title, with a line naming the session, its model and its start. - JSON is
{"id", "title", "project", "model", "created", "messages": [...]}. A message holdsrole,contentandtime(milliseconds since the epoch), and what its transcript line has:meta(true for a reminder of the agent's),nameandtool_call_idfor a tool's result,reasoning_content,calls([{"id", "name", "arguments"}], the arguments as a JSON string),prompt_tokensandoutput_tokens. The messages are the conversation the transcript ends with, those from before a compaction first. - Exit status 1 when the session is not there or the file cannot be written, 2 for a usage error.
In oklm/harness/session.h:
oklm_session_open_read() (nothing is written, so a session
an agent is running can be read), oklm_session_messages(),
oklm_session_search() (a text in the messages and the
calls, whatever the case of its ASCII letters: a hit a line),
oklm_session_export(), oklm_session_remove(),
and the model, start, size and turns in
oklm_session_list()'s entries.
tests/harness/test_session.c (test_read_only)
and tests/harness/test_cli_sessions.c (the command, on a
session made by a scripted agent).
- At start, sessions not written to for
cleanupPeriodDays(30) are removed with their copies; 0 turns the transcripts off.
tests/harness/test_session.c: the conversation read back
is the same, message by message, and the work goes on from it; a line
cut short; a transcript ending with a call and no result; a process
killed (SIGKILL) between a Read's result and the Write after it, then
resumed in another process, which finishes the task; the files restored
by rewind (to before any message, those created removed), the
conversation rewound into a branch the transcript then ends with, and
both after resuming; compactions and clears; a fork; the list, the
latest, the cleanup; the Markdown export.