Native window and API client
oklm-gui, the native graphical tools (Allegro 5 and
nilorea's n_gui) in one program: the launcher, the chat window, the
model manager, the server manager, the conversion tool, the model
inspector, the training window, the sessions browser and the retrieval
manager; and the HTTP client they talk to the server with (server.md).
Status: implemented: the client (plain and TLS, keys, whole answers
and streams, cancellation) and the nine tools, on Linux and Windows (the
MinGW build with the Allegro project's release,
make windows-deps; see build.md).
On Windows the registry of background processes (server.md) and the agents'
listener are there since 2026-11-13, so the tools start, list, follow
and stop their commands as on Linux. Every tool was run with the Windows
build under Wine 11 on 2026-11-13, not on Windows itself: each tool's
section says what was run there, what was not and what differs, and
tools/check_gui_wine.sh replays the flows of each with the
tools' own presses, no place of a window clicked (testing.md). The other tools of the
roadmap (roadmap.md) are to come.
Each tool has a section below, in the order the launcher lists them,
and each section has the same parts: what the tool is for, how it is
started, the window, worked examples, the controls, the options, the
limits. The pictures and the figures of the examples come from runs on
one machine (16 threads, --backend cpu), started in a
folder holding models/, text/,
docs/ and project/; the paths in them are that
folder's. Where a picture was taken with options that put the window in
the state a step describes (--select, --run,
--start), the step says so.
The tools
One program holds every tool, and --tool NAME chooses
one:
oklm-gui # the chat window (the default tool)
oklm-gui --tool chat -m qwen3-8b
oklm-gui --tool models # the model manager: download, list and delete the catalogue's models
oklm-gui --tool servers # the server manager: start, list, stop, restart, follow, resize
oklm-gui --tool convert -m model.gguf --type Q4_K_M # convert or requantize a model
oklm-gui --tool inspect -m model.gguf # the model inspector on a file
oklm-gui --tool train # the training window: a run started and followed
oklm-gui --tool sessions # the agent sessions of a project folder
oklm-gui --tool rag # the retrieval manager: build, check, search and evaluate an index
oklm-gui --tool launcher # the home window: a button for each tool, the documentation
oklm-gui --tool list # the tools' names, one a line
| Tool | Menu entry | What | Section |
|---|---|---|---|
launcher |
OKLM | the home window: a button for each tool, and the documentation | The launcher |
chat |
OKLM Chat | the chat window, the default tool | The chat window |
models |
OKLM Models | the catalogue's models: download, list, delete, open a tool on one | The model manager |
servers |
OKLM Servers | start oklm serve in the background, and every server of
the machine |
The server manager |
convert |
OKLM Convert | a checkpoint or a GGUF file made into a GGUF file of another type | The conversion tool |
inspect |
OKLM Model Inspector | a model as a map, where its layers are and the memory they take | The model inspector |
train |
OKLM Training | a form for each kind of training run, the run followed | The training window |
sessions |
OKLM Sessions | the agent sessions of a project folder | The sessions browser |
rag |
OKLM Retrieval | a retrieval index built, checked, searched, evaluated and attached | The retrieval manager |
The menu entries are those a package installs (build.md), one a line of
packaging/gui-tools.txt.
Every tool takes these options, besides its own:
| Option | What |
|---|---|
--tool NAME |
the tool; --tool list prints the names, one a line |
--size WIDTHxHEIGHT |
the window's size at start, 640x400 at least. Without it each tool has its own: 1100x720, but the launcher (760 wide, a row a tool) and the model manager (980x640) |
--screenshot FILE.png |
save the window to the file and exit: a second after the start, or when the tool says it shows what was asked for. For checks without a person, under Xvfb |
-h, --help |
the tool's usage, then these options |
Every control says what it does when the pointer rests on it (a tooltip), and an empty field shows in grey what goes in it or the default that applies.
The launcher
The home window: it lists the other tools, each with a line saying what it is for and a button that starts it, and opens the documentation. It is for starting from a menu or a double click, where there is no command line to name a tool on; the AppImage, which has no menu, opens on it.
oklm-gui --tool launcher
Menu entry: OKLM.

Example: start a tool.
- Start
oklm-gui --tool launcher. - Press Open Models. The line beside
Documentation says
Models started, in a window of its own.and the model manager opens: another process,oklm-gui --tool models. - Close the launcher. The model manager stays: the tools started live on when the launcher ends.
| Control | What |
|---|---|
| Open NAME, one a tool | starts the tool as a process of its own: the same program again with
--tool NAME |
| Documentation | opens the HTML documentation that came with the program in the system's browser; the line beside it says which file was opened, or that there is none with this copy |
The launcher has no option of its own.
Where the documentation is looked for, from the executable's directory, where each package puts it, then in a source tree:
| Where | Which copy |
|---|---|
../share/doc/oklm/html/index.html |
the AppImage (usr/bin beside
usr/share) |
../docs/index.html |
the .deb (/opt/oklm/bin beside
/opt/oklm/docs) |
docs-html\index.html |
the Windows folder |
../../doc/docs-html/index.html,
build/doc/docs-html/index.html |
a source tree, after make docs-html |
/opt/oklm/docs/index.html |
an installed .deb, last |
From an AppImage. The program runs from a mount the
AppImage makes for the process it starts, and which goes with it. So a
tool is started from the AppImage's own file
($APPIMAGE --gui --tool NAME), which mounts itself again
and lives as long as its window; and the documentation is copied once to
$XDG_CACHE_HOME/oklm/docs-html-VERSION
(~/.cache), again when the AppImage's is newer, and opened
from there, so that its pages are still there when the launcher is
closed.
--press open:TOOL and --press docs are a
test aid: the Open button of a tool's row
(open:sessions) and Documentation pressed
without a person, each in its turn once the window is shown, what the
window says after written on the error stream
(oklm-gui: said: Sessions started, in a window of its own.,
then oklm-gui: pressed: open:sessions); with
--screenshot the window is saved after the last one and
ends, the tools it started staying (The presses).
Limits: the launcher starts tools without arguments; a tool's options
are given on a command line. On Windows it is the same window, the tools
started with oklm-gui.exe --tool NAME. Run on 2026-11-13
with oklm-gui.exe under Wine 11 and Xvfb, from a copy of
the folder make windows stages with the documentation
beside it as the installer puts it (docs-html\index.html),
nothing else on the path: each of the eight Open
buttons started its tool, a process and a window of its own, which
stayed when the launcher was closed; Documentation
found the page, and Wine handed
file:///.../docs-html/index.html to the machine's opener
(xdg-open: the browser showed the documentation's first
page). What Windows itself does with the page
(ShellExecute) was not run. The folder of
make windows alone holds no documentation: the installer
adds it, and the button says when it is missing.
The chat window
A conversation with a model, drawn as Markdown and streamed, with the
server's figures beside it. It is the window to use rather than
oklm chat when the answers hold tables, code or lists to
read and copy, when the context has to change in the middle of a
conversation, or to watch and answer an agent that runs without a
terminal.
oklm-gui -m models/qwen2.5-0.5b/qwen2.5-0.5b-instruct-q4_k_m.gguf
oklm-gui -m qwen3-8b # a model installed by `oklm models pull`
oklm-gui # nothing loaded: the window lists the servers of this machine and the installed models
oklm-gui --url https://host:7430 --key KEY [--ca ca.pem | --no-verify]
oklm-gui --url http://127.0.0.1:8080 --key KEY # a server that is not OKLM's (llama.cpp's, any OpenAI-compatible one): the window says so and goes without what it lacks
oklm-gui --tool chat --agent 7481 # attached to a listening agent
oklm-gui -m qwen3-8b --rag docs.gguf # each message answered from a retrieval index
oklm-gui -m smolvlm-256m --attach photo.png # a picture goes with the first message
oklm-gui -m qwen2-vl-2b --attach clip.ogv # a video: its frames sampled by the server (--ffmpeg for an MP4)
oklm-gui -m qwen3-8b --attach report.pdf --attach notes.docx # documents, read by the window
oklm-gui -m qwen3-8b --attach book.pdf --embed-model bge-small-en-v1.5 # longer than the context: searched for each message
oklm-gui -m qwen3-8b --attach book.pdf --rerank-model ms-marco-minilm-l6-v2 # the passages found put in order by a reranker (an installed one is used without the option; --no-rerank: none)
oklm-gui -m qwen3-8b --attach book.pdf --no-doc-cache # index it anew in a temporary file (by default its index is kept in ~/.cache/oklm/docs for the next window)
oklm-gui -m model.gguf --mmproj mmproj.gguf # a projector for a model that is not in the catalogue
oklm-gui -m qwen3-8b --transcribe-model whisper-small --transcribe-language fr # Record and /listen, in French
oklm-gui -m qwen3-8b --transcribe-model whisper-small --transcribe-translate # what is said in any language, written in English
oklm-gui -m qwen3-8b --mic alsa_input.usb-headset.mono-fallback # another microphone than the system's default (Speech... lists them)
oklm-gui -m qwen3-8b --listen-silence 2 --listen-send # a recording ends after 2 s without a word, and is sent
oklm-gui -m qwen3-8b --no-partial # no words shown while a recording is still coming
oklm-gui -m qwen3-8b --transcribe-attached --attach talk.ogg # an attached recording transcribed without a press
oklm-gui -m qwen3-8b --no-transcribe # no Whisper model beside the chat model
oklm-gui -m qwen3-8b --read-aloud # every reply read aloud, with the installed speech model
oklm-gui -m qwen3-8b --speak-model outetts-0.2-500m # the speech model loaded with the window's server
oklm-gui -m qwen3-8b --speak-model soprano-1.1-80m # Soprano in its place: one voice, 32 kHz, faster than it is heard on a CPU
oklm-gui -m qwen3-8b --speak-model tts.gguf --speak-vocoder vocoder.gguf --speak-voice speaker.json
oklm-gui -m qwen3-8b --no-speak # nothing is read aloud
oklm-gui -m qwen3-8b --chat-file chat.json # a saved conversation opened, to go on with
oklm-gui -m qwen3-8b --talk # a spoken conversation: speak, pause, hear the reply; Escape leaves
oklm-gui -m qwen3-8b --talk --listen-silence 2 --listen-threshold -30 # a longer pause ends a turn; a noisier room
oklm-gui -m qwen3-8b --talk --no-interrupt # with loudspeakers: speaking does not stop the voice
oklm-gui -m qwen3-8b --speak-voice ~/.config/oklm/voices/anna.json # the voice chosen at start
oklm-gui --url http://host:7430 --speak-voice anna --speak-speed 1.25 # one of that server's voices, by its name, a little faster
oklm-gui -m qwen3-8b --read-aloud --speak-whole # a reading is played once its whole sound is made: no bursts on a CPU
oklm-gui -m qwen3-8b --model-settings # the model settings open at start: backend, layers on the device, context, cache, sampling
Menu entry: OKLM Chat. Started with neither
-m, --url nor --agent, as a menu
entry or a double click does, the window loads no model: it asks what to talk to, a server
already running on this machine or an installed model (models.md); with none installed and no server
running it opens on its Models window.

There are two modes, and they behave the same, since every request goes through the HTTP API either way:
- With
-m, the window loads the model and serves it itself on a free loopback port (from 7431). - With
--url, it talks to a server elsewhere: anoklm serve, or a server that is not OKLM's, which is asked for nothing but models and replies.
Started with nothing named
oklm-gui alone, the menu entry, the launcher's
Chat and a double click name no model and no server.
The window then loads nothing by itself: a server of this machine may
hold a large model already, and a second copy would take the memory
twice and leave the window's own cache what is left. It opens on a
window that asks What should this window talk to?

| Control | What |
|---|---|
| Servers running on this machine | what answers on a loopback port from 7430 to 7480, found as Connect... finds it, each with its model's name |
| Saved servers | the servers connected to before, on this machine or elsewhere, as the Connect window keeps them with their key and their certificate settings; the heading is left out when there is none |
| Agents | the agents started with oklm agent --listen, as the
Connect window lists them; left out when there is none |
| Installed models, loaded by this window | the chat models of the catalogue that are installed, with their size and the context they were trained with |
| A model's file, loaded by this window | Choose a file... opens the system's file dialog for
any GGUF file, a model outside the catalogue included. The file chosen
is listed under it, and under the file its projector:
none, until that row's button chooses one in the file dialog, as
--mmproj names one. A file that is an installed catalogue
model's is loaded as that model, with the catalogue's projector, and has
no projector row |
| The text on the right | what the selected row is, and what a press does or came to: a server's model is loaded already and the window loads nothing; a model is loaded by the window, beside what the servers listed hold; why a model was not loaded |
| The button under it | Connect to this server links the window to the selected server, as Connect does, and nothing is loaded: a saved server with the key and the certificate settings kept with it. A server that asks for a key, or refuses the one kept, is handed to the Connect window, its address and what is kept in the fields, where the key is typed. Attach to this agent attaches as Connect does. Load this model and Load this file load the model in this window (below). Choose a file... and Choose its projector... open the file dialog. Without a row selected it does nothing |
| Refresh | looks again for servers, agents and installed models |
| Models... | the Models window: the whole catalogue, to download a model and use it |
| Connect... | the Connect window: a server by an address typed, with its key and its certificate; a session to read |
The servers and the agents are those of the Connect window's own search (one search fills both lists); nothing else is looked for.
A model chosen there is loaded in the window
Load this model and Load this file
do not start the window again. The settings the model would be loaded
with are the window's (its command line's) with what is kept for that model, as a start with
-m has them; the memory line
judges them first, as Model settings does before a
restart and as a model loaded in place of another is judged:
- settings that do not fit, also without what is kept for the model,
load nothing. The text on the right gives the line's reason
(
`tiny.gguf` is not loaded: with a context of 16384 tokens it does not fit: that cache holds 0.5 contexts, less than one; a context of 16384 tokens takes 2 MiB. Nothing was loaded: choose a smaller model, or start the window with a smaller context (-c) or fewer layers on the device.) and the window that asks stays open; - settings that fit close the window that asks and load the model on a
thread of its own, the status line reading
Loading NAME in this window, with the window's settings. The window goes on; nothing is sent until it is done.The window's own server is then made in place, on a free loopback port, and the status line says what it was loaded with (The window now serves ...). What was typed in the message field stays, and so does the conversation.
The models beside it are found as a start with -m finds
them: the catalogue's projector for a catalogue model, the projector
chosen for a file, the transcription and the speech models that are
installed (unless --no-transcribe, --no-speak
or an option names others); a Whisper model or a catalogue projector
that cannot be opened is left out, as at a start. A model that cannot be
served after all (a file that is no model) leaves the window serving
nothing, and the window that asks opens again with the reason. In a
window that is linked to something already, the button starts the window
again on the model, as Use this model does in a window
with no server of its own.
Nothing is selected when the window opens, also when one row is all there is: the person presses. With no chat model installed and no server answering, the Models window opens instead, where a model is downloaded.
Until something is chosen the window is linked to nothing, and says
so: the status line reads
Nothing is loaded: choose a server, an agent, an installed model or a model's file; Connect... and Models... have the rest.,
the side panel no model and no model served,
the Context slider and Apply context are off, and the
model settings show No model, with everything but the
sampling locked. A message sent then stays written, the window that asks
is opened again if it was closed, and the status line reads
Not sent: nothing is loaded yet. Choose a server, an agent, an installed model or a model's file first, or press Connect... or Models...; the message stays written.
(a scripted --send writes the same on the error stream,
after oklm-gui: --send:). The commands that need a model
answer with a note (No model yet: ...);
/model NAME and /connect URL choose from the
message box. -m, --url and
--agent are as before: the model is loaded at once, the
server or the agent connected to, and this window is not shown; the
server manager's Chat passes --url and so
loads nothing either.
Example: a conversation with the window's own model
- Start the window on an installed model, on the CPU:
oklm-gui -m llama3.2-1b --backend cpu. The side panel names the model and the address the window serves it at,http://127.0.0.1:7431, and the line under Apply context readsA context of 4096 tokens: a cache of 512 MiB in memory, with the memory that is free. - Click the message box, type
In two short bullet points, say what a shell script is. Then show a three-line example in a code block.and press Ctrl+Enter (a plain Enter starts a new line). - The reply streams in under the model's name, its code block in a
frame and its inline code in the fixed face. When it ends the status
line reads
111 tokens in 3.3 sand, within four seconds, the side panelGenerated: 111,Prompt tokens: 59,Requests: 1: the picture above (taken with a seed set in the model settings, henceSampling: seed 7: the same reply at each run). - Drag the Context slider to 16384. The value gets a
*(not applied yet), and the line readsA context of 16384 tokens: a cache of 7.0 GiB in memory, in grey: it fits. At 65536 it is red:a cache of 28.0 GiB in memory ... Larger than the 40960 tokens the model was trained with: it may read badly beyond.(These figures and the two pictures below are Qwen3-0.6B's, whose cache is 1.75 GiB at 4096 tokens and which was trained with 40960; Llama 3.2 1B takes 2.0 GiB at 16384 and was trained with 131072.) - Back at 16384, press Apply context. The status line
says
Context: 16384 tokens. The conversation is kept; the model reads...and the panelContext: 16384 per sequence; the conversation is still on screen, and the model reads it again on the next message.


Example: a server found on this machine, then back
A server runs on the machine, started in a terminal with
oklm serve -m qwen3-0.6b --backend cpu --port 7440, and an
agent listens in the folder the window was started in
(oklm agent -m smollm2-135m --backend cpu --listen).
- Press Connect.... The list has the server under
Servers on this machine
(
127.0.0.1:7440 Qwen3-0.6B-Q8_0), the agent under Agents with its port, model and project, and the sessions of the folder. - Click the server's row: its address goes in Server address.
- Press Connect. The window closes, the side panel
now shows
http://127.0.0.1:7440and that server's figures, and the conversation is still on screen. - Type
And in one sentence, what does chmod +x do?and press Ctrl+Enter. Qwen3 answers, its reasoning dimmed as a quote above the answer; the status line reads160 tokens in 4.0 s. - Press Connect..., then Use my own
model. The status line says
Back on the window's own model. The conversation is kept.and the panel showshttp://127.0.0.1:7431again.


To attach to the agent instead, click its row and press Connect: the window shows the agent's events from its start, the message box sends to it, and a tool call that asks for permission brings Allow, Always and Deny beside the status line (Connecting).
Example: commands
The window of the first example, on Llama 3.2 1B.
- Type
/sysin the message box. A list opens above it with the commands that start so, here one:/system [TEXT|-],Show the system prompt, or set it and start a new conversation (- for none). Press Tab: the message becomes/system. TypeAnswer in one short sentence.and press Enter (a command is run by a plain Enter). The conversation is emptied, and a note says, dimmed under the command's name,The system prompt is set (29 characters) and a new conversation starts.; the side panel gainsSystem prompt: Answer in one short sentence.under the server's figures. - Type
What is a shell script?and press Ctrl+Enter. The reply is one sentence (46 tokens). - Type
/temp 0and Enter: the Temperature slider goes to 0.00 and the note readsTemperature: 0.00 (the likeliest token each time). Then/retryand Enter: the reply is dropped and written again, at that temperature. The note stays where it was written, above the new reply, and the model is never sent it. - Type
/re: the list offers/reset,/retryand/rewind [N], each with its line of help. This is the picture below. Down twice highlights/rewind [N], Enter takes it (/rewind), Enter again runs it: the conversation is back to before the question, which is in the message box again, to change and send. - With a conversation on screen,
/statswrites its messages and tokens, the context's use, the last reply's tokens and time and the side panel's figures in a note;/save chat.jsonwrites it whereoklm chat -m MODEL --resume --chat-file chat.jsoncontinues it, and whereoklm-gui --chat-file chat.json, or the file dropped on the window, opens it again;/exportasks for a file and writes it as Markdown.

Example: a quantized cache and another sampling, in the middle of a conversation
- Start
oklm-gui -m smollm2-135m --backend cpu -c 1024and sendhello. - Press the model's name at the foot of the side panel,
SmolLM2-135M-Instruct-Q8_0. The Model settings window opens: Backendcpu, Layers greyed withCPUbeside it, Cacheoff (f16), Context 1024, and under them1024 tokens on cpu, a cache of four contexts shared by the requests: 342 MiB of memory (weights 136 MiB, cache 90 MiB), ... It fits. - Type
0.3in Temperature and7in Top-k. Nothing is asked: the side panel's Temperature slider goes to 0.30, its figures gainSampling: top-k 7, and the next request carries both. - Choose
8-bit (q8_0)in Cache. Half a second later the line reads..., its keys Q8_0 and its values Q8_0: 299 MiB of memory (weights 136 MiB, cache 47 MiB), and the line above the buttonsChanged, not applied: -ctk -ctv. Apply asks, then restarts the model with them. - Press Apply. A window asks
Restart the model with these settings? The conversation is kept; the model loads again (it took 0.1 s last time).and names what changes,-ctk -ctv, with Restart, Cancel and the box Do not ask again in this window. - Press Restart. The status line says
Restarting the model with the new settings, thenThe model was restarted in 0.3 s with these settings (context 1024, -ctk q8_0 -ctv q8_0). The conversation is kept; the model reads it again on the next message.helloand its reply are still there; sendagain: the reply's prompt holds both turns. - The line above the buttons now reads
The model runs with these settings. Kept for it, for the next window: -ctk -ctv --temp --top-k. Defaults forgets them.Close the window and start it again the same way: the cache is in 8 bits and the sampling 0.3 and 7 without a press. Defaults asks for the restart back tooff (f16), and the model has nothing kept again.


Example: a picture, then a document
SmolVLM-256M is installed
(oklm models pull smolvlm-256m): a small vision model whose
projector the catalogue has. red-ball.png is a drawing of a
red disc on a white ground, harbour.pdf a page of a
harbour's timetable: both are in the repository, under
tests/fixtures/gui (tools/docs/gui_fixtures.py
makes them).
- Start
oklm-gui -m smolvlm-256m --backend cpu. The window serves the model with its projector; nothing shows it yet. - Drag
red-ball.pngfrom the file manager onto the window (or press Attach and choose it, or type its path in the field beside the button and press Enter). A box appears above the field, with the picture in small,red-ball.pngandPNG, 949 bytes; the status line saysAttached: red-ball.png PNG, 949 bytes. It goes with the next message. - Type
What is in this picture? Answer in two sentences.and press Ctrl+Enter. The box leaves the row; the conversation shows the picture in small under You, its name and size on a line, then the question. The model answersA red circle is in the background. - Type
harbour.pdfin the field beside Attach and press Enter. The window reads the PDF itself; its box saysPDF, 1 KiB, 157 tokens. A click on the box says the same in the status line, thexon it would take it out. This is the picture below. - Type
When does the ferry leave?and send. The conversation shows one line for the document,harbour.pdf PDF, 1 KiB, 157 tokens, not its text; the model was given the text under the file's name.

On a text model the same picture is refused before anything is sent.
With oklm-gui -m smollm2-135m, its box is framed in red
(cannot be sent: click for why), and Send leaves the
message where it is and says
red-ball.png cannot be sent: this model has no projector, it reads text only (a vision or audio model of the catalogue has one, or start the window with --mmproj FILE). Remove it (the x on it) to send the message.
A document longer than the context is searched, not cut.
town.md is a text of 26872 tokens, twenty sections each
about one place of a town and naming its keeper once; an embedding model
is installed (oklm models pull bge-small-en-v1.5, 127
MiB).
- Start
oklm-gui -m qwen3-0.6b --backend cpu, attachtown.mdand sendWho is the keeper of the observatory? Answer in one sentence.The message leaves its box and the status line saysIndexing town.md to search it, since it is longer than the context: 64 passages embedded with bge-small-en-v1.5, 0:21. Stop cancels. - A note headed Documents is written in the
conversation:
town.md is longer than the context leaves room for: it is not cut, it is searched. It was indexed here in 41.3 s (100 passages, with bge-small-en-v1.5), and 11 of them, the ones found for this message, went with it., then what that means for the messages to come. Under You the document's line readstown.md Markdown, 117 KiB, 26872 tokens (searched: 11 of its 100 passages sent), and the model answersThe keeper of the observatory is Jonas Pike.: the sentence is on line 813 of 1240. - Ask
And who is the keeper of the ferry?Nothing is indexed again: the document is searched for this message and the answer isThe keeper of the ferry is Tobias Lund. - Close the window, start it again and attach
town.mdagain: the note now readsIts index was found in the cache, built by an earlier window: nothing was embedded again, and the message goes at once. With a reranker installed (oklm models pull ms-marco-minilm-l6-v2) the note endsput in order by the reranker ms-marco-minilm-l6-v2.
The picture below is the same conversation with SmolLM2-135M at a
context of 2048 and --embed-model hash, which needs no
model and indexes the document at once, on the repository's
tests/fixtures/gui/town.md: the same twenty sections in 51
KiB and 12147 tokens (60 passages, 6 of them sent with each message),
where the figures of the steps above are those of a longer text of the
same shape.

Without an embedding model the window says so and offers the cut:
The document holds about 26872 tokens and 1476 fit in the context of 2048 (the conversation holds 0, the reply keeps its share). Not searched instead: no embedding model is installed: Models has bge-small-en-v1.5 (127 MiB) under Retrieval, or name one with --embed-model. Send the beginning that fits, or cancel.
and two buttons show beside the line. Send the
beginning sends what fits, and the conversation's line says how
much (the beginning sent: 15934 of 140037 bytes);
Cancel sends nothing and leaves the message and the
file, to raise the context first.
Example: say it instead of typing it
Whisper base is installed
(oklm models pull whisper-base, 281 MiB) and a microphone
is plugged in.
- Start
oklm-gui -m qwen3-8b. The side panel's figures end withSpeech to text: whisper-base: the window's server was started with it, the smallest installed Whisper model of the catalogue. - Type
Summarize in one line:and press Record, beside Attach. The button reads Stop; where the path's field was, a red dot, the time (0:05) and the microphone's level, green, amber in its last fifth when the voice is close to saturating. The status line saysRecording. Record again, or Enter, stops it and writes what was said in the message; Escape cancels.This is the picture below. - Speak, then press Record again (or Enter). The row
says
Transcribing the 0:11 recorded..., and three seconds later the message readsSummarize in one line: And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.with the cursor in it. The status line:Transcribed 11.0 s of sound in 3.3 s (by the window's server, with whisper-base): the text is in the message, to read and correct before it is sent. - Correct what was misheard, and send as any message (Ctrl+Enter).

The same from the keyboard: /listen starts,
/listen stop (or /listen again, or Enter)
ends, and /listen send records then sends the message as
soon as the text is in it. Escape while recording keeps nothing; Escape
while the sound is transcribed drops its text.
A recording made elsewhere: drop talk.ogg on a window
whose model reads text only. Its box says
not heard by the model and has a
Transcribe button; a press writes
Transcript of talk.ogg: and the text under it in the
message, and the box leaves the row: the sound itself is not sent.
Example: have the reply read aloud
The speech model is installed
(oklm models pull outetts-0.2-500m, 636 MiB, its licence
asked) and the machine has a sound device.
- Start
oklm-gui -m smollm2-135m. The side panel's figures end withRead aloud: off, outetts-0.2-500m: the window found the model and has not loaded it. - Ask
Why is the sky blue? Answer in two sentences.Beside the model's name, over the reply, a small Speak shows once the reply ended. - Press it. It reads Stop, in amber; where the path's
field was, the row says
Loading the voice (outetts-0.2-500m)for a moment,Part 1 of 1: the first sound is made, thenSpeaking, part 1 of 1with the time since the press and the sound made so far (0:12 3.5 s of sound), and Stop reading is at the row's end. The status line:Reading the reply aloud (the window itself, with outetts-0.2-500m): the first sound comes once its first words are made, some seconds on a CPU.This is the picture below. The conversation goes on meanwhile: a message can be typed and sent. - Let it end
(
Read aloud: 16.1 s of sound made in 43.9 s (by the window itself, with outetts-0.2-500m).), or press Stop reading, the reply's Stop, or Escape: the voice ends at once and the status line saysThe voice is stopped.Pause, beside Stop reading, holds the voice where it is and reads Resume; the rest of the sound goes on being made meanwhile. Press Speak on the same reply once it was read to its end:Heard again: the 16.1 s of sound of its last reading were kept, nothing is made anew. - Tick Read aloud, beside Record (or
type
/speak on). The next reply is read by itself while it is written: its first sentence goes to the voice as soon as it is whole, and the row saysSpeaking while the reply is written, sentence 2. One cut by Stop while it is written is said up to its last whole sentence. /speak The meeting is at three.reads a text that is no reply.

Started with
oklm-gui -m smollm2-135m --speak-model outetts-0.2-500m,
the window's own server loads the speech model with the chat model and
speaks itself (the window's server, with outetts-0.2-500m):
the first reading does not wait for the model, and the 1.6 GiB are taken
from the start.
Example: a spoken conversation
Whisper base and the speech model are installed, a microphone is plugged in and headphones are on (with loudspeakers, see A spoken conversation).
- Start
oklm-gui -m qwen2.5-0.5b --backend cpuand tick Talk, beside Read aloud (or type/talk, or start with--talk). The path's field gives way to the mode's row: a green dot,Listening, the time, the microphone's level,Escape leaves. The status line saysTalk is on: speak, and after 1.2 s of silence what was said is sent; the reply is spoken while it is written, and speaking over it stops it. Escape, the Talk box or /talk off leaves., and the side panel gainsTalk: on, interrupted by speaking. Record is dimmed: the microphone is listened to already. - Ask
Why is the sky blue? Answer in three short sentences.While you speak the row readsListening: you speak. Stop speaking: 1.2 seconds later it readsTranscribing what was said, in amber, then the question is in the conversation under You, as Whisper heard it, and the row readsThinking, in blue. Nothing was pressed. - The reply is written. Its first sentence goes to the voice once it
is whole, and when the first sound is made the row reads
Speaking: speak to interrupt, in violet. This is the picture below. - Speak over the voice:
Stop. Tell me what two plus two is.A quarter of a second into your first word the voice stops, and the reply too were it still being written; the status line saysYou spoke: the voice and the reply are stopped, and what you say is the next message., and the row is back toListening: you speak. At your pause the new question is transcribed and sent, and its reply spoken. - When the voice has finished the row reads
Listeningagain: the next turn is yours. Press Escape (or untick Talk, or type/talk off):Talk is off: the microphone is closed.

The same conversation was run on a machine with no microphone and no loudspeaker, the two questions being recordings fed as the microphone and the voice written to a file (the test aid). What the window printed, on a 16-thread CPU that other work shared:
$ oklm-gui -m qwen2.5-0.5b --backend cpu --talk --mic-file sky.wav --mic-file two.wav@voice+3 --speak-file said.wav
oklm-gui: talk 0.0 s: listening
oklm-gui: talk 4.4 s: a turn of 4.1 s ended
oklm-gui: talk 4.4 s: transcribing
oklm-gui: transcribed 4.1 s in 0.8 s by the window's server, with whisper-base: Why is the sky blue and some three short sentences?
oklm-gui: talk 5.2 s: thinking
oklm-gui: the first sound came 10.6 s after the message was sent (its first sentence was written after 0.6 s)
oklm-gui: talk 15.8 s: speaking
oklm-gui: talk 19.8 s: listening (the reply was spoken over)
oklm-gui: the voice was stopped after 1.7 s of sound made in 14.6 s by the window itself, with outetts-0.2-500m
oklm-gui: talk 22.3 s: a turn of 3.1 s ended
oklm-gui: talk 22.3 s: transcribing
oklm-gui: transcribed 3.1 s in 1.2 s by the window's server, with whisper-base: Stop, tell me what 2 plus 2 is.
oklm-gui: talk 23.5 s: thinking
oklm-gui: the first sound came 6.8 s after the message was sent (its first sentence was written after 0.2 s)
oklm-gui: spoke 1 part, 1.7 s of sound in 6.9 s by the window itself, with outetts-0.2-500m
oklm-gui: talk 30.4 s: speaking
oklm-gui: talk 30.4 s: listening
$ oklm transcribe -m whisper-base said.wav
The sky is blown because 2 plus 2 is 4.
The two recordings were made by the speech model itself
(oklm speak), which is why Whisper hears
and some for Answer in. The second one starts
three seconds after the first sound of the first reply
(@voice+3): a quarter of a second later the level has held,
the voice is stopped, and the file holds the five words that were said
of the first reply (The sky is blue because, as Whisper
hears them), then the second reply whole. The first reply was already
written when it was spoken over, in 8 seconds; a longer one would have
been cut there.
The conversation
| Control | What |
|---|---|
| The conversation | each message drawn as Markdown: emphasis, <think> block is dimmed, as a quote, and so is the
reasoning a server gives apart from the answer
(reasoning_content: gpt-oss's analysis channel), which is
not read aloud, not sent back with the next request, and written by
/save and /export as other models' reasoning
is. A reply streams in as it is written, and the view follows the end
unless the scrollbar was moved up |
| Selecting | drag within a message or across several, double-click a word. Ctrl+C
copies the selection as plain text (when the message box does not have
the keys); a selection is also the X11 primary selection, for a
middle-click paste. A right click copies the whole message as Markdown,
without a reply's <think> block |
| Links | a click on an http(s) link opens it in the system's
browser (xdg-open, open or ShellExecute, the URL passed as one argument,
no shell); a press that drags out a selection opens nothing |
| The message box | the message; Enter starts a new line, Ctrl+Enter or Shift+Enter
sends. A line that starts with / is a command, which a
plain Enter runs (Commands) |
| The list above the message box | shown while a command is typed: the commands that start with what is typed, then the values of its argument; Up and Down choose, Tab or Enter completes, Escape closes |
| A note | a command's answer: its name as the heading, the text dimmed. It is not part of the conversation the model reads |
| Attach | opens the system's file dialog (several files can be chosen), or attaches the file whose path is typed in the field beside it (Files with a message) |
| The path's field | a file's path; Enter attaches it. A file:// address is
taken too |
| The attached files | shown above Attach while there are some: a box a
file, with an image's thumbnail (a video's is its first frame) or
IMG, SND, VID, DOC,
its name, what it is, its size, a video's length and a document's
tokens; framed in red with the reason when it cannot be read or sent. A
click says everything about it in the status line; its x
removes it |
| Download or Agree and download, Cancel | shown when /model named a chat model that is not
installed: download it and serve it, or not (Commands) |
| Send the beginning, Cancel | shown when the attached documents are longer than the context leaves room for and cannot be searched instead (no embedding model, or the search failed or was stopped): send what fits, or nothing |
| Send | sends the message |
| Stop | cuts the reply short (the server drops the request), or interrupts the attached agent's work |
| Speak beside a reply's name, Read aloud, Pause, Stop reading | a reply read aloud (again from its kept sound once it was read), every new reply while it is written, the voice held and going on, the voice stopped (Replies read aloud) |
| Talk | a spoken conversation: the microphone is left open, what is said is sent at a pause, the reply is spoken, and the window listens again (A spoken conversation) |
| Voice: | the voice replies are read with: one of the voices of the server that speaks, by its name, or the built-in one or a speaker file when the window speaks itself (the voice) |
| New chat | clears the conversation and starts another |
| The status line | the tokens and the time of the last reply, the server's error, or what the last press did |
| Allow, Always, Deny | shown while an attached agent asks permission for a tool call: run it, run it and the calls its rule covers from now on, refuse it |
Commands
A message that starts with / is a command. It is read
with the table every chat shares (inference.md, where the
full list is, chat by chat), runs in the window, and is never sent to
the model. Its answer is a note in the conversation:
the command as its heading, the text dimmed as a reply's reasoning is. A
note is the window's, not the conversation's: no request carries it,
/copy, /save and /export leave it
out, and New chat clears it with the rest.
//text, or a blank before the slash, sends a text that
starts with /; a path is a message too
(/usr/bin/cc fails).
While the command's name is typed, a list above the message box shows
the commands that start with it (by another of their names too:
/ctx), each with its arguments and its line of help. Once
the name is followed by a blank it shows the values of the argument,
where they are known: on and off,
markdown and json, the installed chat models
for /model, the commands for /help,
off for /rag, and on,
off and stop for /speak, whose
argument is otherwise a text, and on and off
for /talk.
| Key | In the list |
|---|---|
| Up, Down | move the highlight |
| Tab | takes the highlighted row: the message becomes that command, or gets that value |
| Enter | completes a name that is not whole, or takes a row that Up or Down chose; otherwise it runs the command |
| Escape | closes the list until the text changes |
| a click on a row | takes it |
A name or an argument that is wrong is answered by a note
(unknown command /modle: did you mean /model?,
usage: /export [markdown|json]) and stays in the message
box, to be put right. What the window cannot do where it is says so and
goes: on a server at an address, /model qwen3-8b answers
/model cannot be done: the model cannot be changed from here.
/model on a model that is not installed
downloads nothing by itself. As the terminal's selection does (inference.md), it asks first, in the window: a
note tells the model's size, its files, the folder they go to and its
licence with the address of its text, and two buttons show beside the
status line.
| The model's licence | The note ends with | The button |
|---|---|---|
| asks for nothing | **Download**, beside the line under the message, starts it. |
Download |
| has to be agreed to (and was not yet) | Using this model means agreeing to that licence: read it first. **Agree and download** ... says yes and starts the download. |
Agree and download: the press is the agreement,
kept as the Models window's box and
oklm models pull --accept-licence keep it |
Cancel downloads nothing. While the download runs
the conversation goes on, the status line says how far it is
(Downloading smollm2-135m: 61 MiB of 138 MiB, 36.5 MiB/s. ...)
and Cancel stops it, what arrived being kept for the
next try. Once the model is there the window's server loads it, as for
an installed one; when a reply is being written at that moment, a note
says to type /model NAME once it ended. A download that
fails leaves its reason in a note. Run on 2026-11-13 with an empty model
folder: /model deepseek-coder-1.3b asked with its licence
and Cancel left the window as it was;
/model smollm2-135m then Download fetched
its 138 MiB, the window started again on it and it answered the next
message.

What the window is linked to decides which commands it has:
- its own server: all of the table below;
- a server at an address: the same, less what that
server cannot do. It tells what it can in
GET /api/commands(server.md): its context is resized when it takes the request, its model and its index are its own; - an agent:
/help,/attach,/detach,/save,/export,/copy,/stop,/listen,/speakand/pauseare the window's (/talkanswers that it is a conversation with a model). Any other line that starts with/is sent to the agent as a message, since the agent has commands of its own (agent-ui.md); - nothing (no model yet, a session being read):
/help,/model,/models,/connect, and the commands that only read or set something.
| Command | In the window |
|---|---|
/help [NAME] |
the commands of what the window is linked to, a line each; or one command in full, with its other names |
/reset, /clear, /new |
what New chat does |
/system [TEXT|-] |
sets the system prompt and starts a new conversation: it goes first
in every request from then on, and its start shows in the side panel.
Without a text, shows it; /system - removes it |
/model [NAME] |
without a name, opens the Models window. With the name of an
installed chat model, what Use this model does: the
window's server loads it in place of the model it serves, and the
conversation goes on with it (a window linked to a server at an address
asks that server, POST /api/model, when it says it loads
another model: the name is then one of the server's machine; otherwise
the command says the model cannot be changed from here; on a server that is not OKLM's the
name is one of those the server lists, carried by the requests from the
next message and kept with a saved server, and without a name the note
says which is asked and lists them). A chat model that is not
installed is asked about first (below):
its size, where it goes and its licence in a note,
Download (or Agree and download) and
Cancel beside the status line. A model that is no chat
model: the Models window opens on it, where it says what it is for. A
file's path is served as -m FILE would |
/models |
the catalogue by group, each model with its size and whether it is installed; before it, on a server that is not OKLM's, the models that server lists |
/context [TOKENS [force]] |
without a number, the size, the size the model was trained with and
the tokens the conversation holds. With one, what the slider and
Apply context do, for any size from 256 to 1 048 576:
refused while a reply is written; a size the memory plan refuses is not
applied, its reason in the note and in red under the slider, and
force applies it anyway. The server makes its cache on a
thread of its own, and the note is written when it is done (the context size). On a server that is not OKLM's the
size is the window's own setting: applied at once, nothing asked of the
server |
/attach FILE |
attaches the file, as the path's field does |
/detach [N] |
empties the row of attached files, or removes the Nth |
/rag INDEX|off |
on its own server: off stops searching the messages in
the index, the open index's name uses it again, another index starts the
window again on it (A retrieval
index) |
/save [FILE] |
writes the system prompt and the messages as JSON, the file
oklm chat --resume --chat-file FILE reads and the window
opens again (--chat-file FILE, or the file dropped on it);
a document is in its message as the model read it. Without a name, the
system's file dialog asks for one |
/export [markdown|json] |
writes the conversation for a person (Markdown: a heading a turn, a
message's files as a list, a reply's reasoning as a quote) or for a
program (JSON: model, system, and
messages with their attachments and
reasoning_content). The system's file dialog asks for the
file; where there is none it is chat-DATE-TIME.md (or
.json) in the current folder |
/copy [N] |
the latest reply, or the Nth before the end, to the clipboard as Markdown, without its reasoning |
/retry, /regenerate |
drops the last reply and has it written again |
/rewind [N], /undo |
removes the user's last message and what followed, or the last N; the message removed last goes back in the message box (its files are not attached again) |
/stop |
what Stop does |
/stats, /usage |
the conversation's messages and tokens, the context's use, the last reply's tokens and time, what the commands set, and the side panel's figures |
/temp [T], /max-tokens [N] |
move their slider, whose range grows for a value beyond it; without a value, tell it |
/top-k [K], /top-p [P],
/min-p [P], /seed [N] |
kept by the window and sent with each request from then on
(top_k, top_p, min_p,
seed), and shown in the side panel; while one is not set
the server's own value applies. Without a value, tell it |
/thinking [on|off] |
asks a model whose template has the switch (Qwen3) to reason before
it answers, or not: chat_template_kwargs with
enable_thinking in each request |
/connect [URL] |
links the window to the server at the address, with the key it was saved with; without an address, opens the Connect window |
/video [fps F] [frames N] | default |
how the videos attached next are sampled by the server:
fps F the frames taken a second at most,
frames N (1 to 32) the frames kept at most, either or both;
default gives the choice back to the server (1 a second, 8
at most); alone it says what holds (A
video). The window's own: the terminal chat attaches no video, the
page sends none of this |
/decide A | B | ... |
has the server choose among the options for the conversation so far
(POST /v1/decide, server.md): a
note with each option's probability and the decision, the conversation
left as it was |
/listen [stop|send] |
records the microphone, as the Record button does
(Speech to text): alone it starts a
recording, or ends the one being made; stop ends it;
send records and sends the message once what was said is in
it. Where nothing transcribes, the note says why |
/speak [on|off|stop|pause|resume|TEXT] |
reads the last reply aloud, as the Speak beside it
does (Replies read aloud);
on and off set Read aloud,
every new reply read while it is written; stop ends what is
being said and the request behind it; pause holds the voice
and resume has it go on, the two sides of
/pause named; anything else is a text to read. The five
words count when they are the whole argument, in any case. Where nothing
reads aloud, the note says why |
/pause |
holds the voice that reads aloud where it is, or has it go on from
there when it is paused, as Pause and
Resume do. Nothing is being read aloud
when there is no voice |
/talk [on|off] |
starts a spoken conversation, as the Talk box does (A spoken conversation), or leaves it; alone it does the one or the other. It needs something that transcribes and something that reads aloud: the note names the one that is missing |
Files with a message
Files go with the next message, eight at most. They come from a drop
on the window (one or several files; dropped text goes in the message
box), from Attach and the system's file dialog, from a
path typed beside the button, from a picture pasted with Ctrl+V (below), or from
--attach FILE on the command line. Each is read on a thread
of its own while the window goes on drawing, and Send waits: it says
... is still being read until it is. Once the message is
sent the row empties and the conversation shows what went: an image in
small with its name, a video by its first frame in small with its name,
length and frames
(clip.ogv Ogg video, 73 KiB, 4.0 s, 4 frames), any other
file as one line with its name, kind and size.
| File | What happens |
|---|---|
| PNG, JPEG, BMP, TGA, PCX, WebP, PPM, PGM | an image: decoded once (for its thumbnail, and to refuse here what
the server could not decode), sent as an image_url part, a
data: URL of base64 (server.md) |
| WAV, MP3, Ogg, FLAC, Opus | a sound: decoded once, sent as an input_audio part to a
model that hears; its box has a Transcribe button,
which writes its words in the message (Speech
to text) |
Ogg Theora (.ogv); MP4, M4V, Matroska, WebM, QuickTime,
AVI, MPEG, WMV, FLV with --ffmpeg |
a video: decoded once with the reader oklm describe
uses (oklm_video_load(), multimodal.md) for its first frame, its
length and the frames its sampling keeps (the server's: a frame a
second, 8 at most; or /video's), and to refuse here what
the server could not decode; sent whole as a video_url
part, a data: URL of base64 (server.md), which the server
samples: one video item for Qwen2-VL and Qwen2.5-VL, its frames as
images for another image projector, its sound track for an audio
projector |
| PDF, DOCX, HTML, JSON | a document: its text extracted by the library's reader
(oklm_ingest_file(), what oklm ingest prints:
multimodal.md) |
| anything else | read as text (Markdown, CSV, code, logs); a binary file is refused |
A saved conversation dropped on the window is
opened, not attached, as the dashboard's chat does: a .json
file that holds an object with messages, each with a
role and a content (what /save
and /export json write, what
oklm chat --chat-file keeps, what the dashboard saves). It
takes the place of the conversation on screen, with its system prompt,
and the status line says
Conversation opened from chat.json: 2 messages and its system prompt. The model reads them with the next message.;
--chat-file FILE does the same at start, and with
--send goes on with it. A reply's
reasoning_content is shown as its reasoning. What such a
file does not keep is not there again: the window's notes, the files
that went with a message (a document is in its message's text as the
model read it; an image or a sound is not). It is opened once a reply
being written ended, and not while the window is attached to an agent. A
JSON file that is no conversation is a document like another.
A video is told by its file's extension
(.ogg is a recording, .ogv a video;
.ts is read as text, source code more often than a
transport stream). Ogg Theora is decoded as it is; any other container
goes through an ffmpeg subprocess, here and in the window's
own server, only when the window was started with --ffmpeg
(or OKLM_FFMPEG=1 is set, as for oklm describe
and oklm serve). Without it such a file is refused when it
is attached, on the window's own server:
clip.mp4 cannot be attached: it is not a video this build decodes (only Ogg Theora (.ogv) is decoded natively: allow ffmpeg with --ffmpeg or OKLM_FFMPEG=1, or convert the file to .ogv).
On a server at an address, which may have been started with
oklm serve --ffmpeg, it is attached without a thumbnail and
the server decides. A video above the server's default limit on decoded
frames (8 frames of 3840 x 2160) is refused with the figures, and one
above 20 MiB as an image or a sound is. It goes to a link that takes
images or sounds: a projector for images reads its frames, one for
sounds its sound track (a video without sound is then refused by the
server, whose message the status line shows).
A video's sampling is the server's unless
/video chose one (2026-11-14):
/video fps 2 frames 16 has every video attached from then
on sent with "fps":2,"frames":16 on its
video_url part (server.md), and decoded by the
window the same way, so the frames its box and its line count are those
the model is shown
(testsrc.ogv Ogg video, 14 KiB, 2.0 s, 3 frames after
/video fps 2 frames 3, 2 frames without). A video keeps the
sampling it was attached under: its part carries it in every later
request of the conversation, and a /video typed afterwards
changes the next videos only, not what the model was shown under an
earlier message. /video alone tells the sampling,
/video default gives it back to the server; a word that is
not understood changes nothing
(Not changed: frames is an integer from 1 to 32, the frames kept at most.).
A video's sound has the recording's
Transcribe button on its box when the file has a sound
track (2026-11-14). A press reads the track with the reader
oklm describe gives a video to an audio model with
(oklm_video_load(), 16 kHz mono) and has it transcribed as
a recording is, by the window's recognizer or the server's: its words
are written in the message under Transcript of NAME:, and
the video stays attached
(clip.ogv is transcribed (2.0 s of sound in 0.1 s, by the window's server, with whisper-small, ...): its text is in the message, to read and correct; the video stays attached.),
so a model that is shown the frames can be told what is said in them.
With --transcribe-attached this is done without a press for
a video on a link that takes its frames. A video without a sound track
has no button.
Images, sounds and videos need a model with a
projector. The window's own server opens one when the catalogue has it
for the model named by -m (smolvlm-256m,
ultravox-1b...), or the file --mmproj names.
Whether the link takes images or sounds is read in the server's figures
when the first one is attached (the vision and
audio members of GET /api/stats, server.md), for the window's own server and for
one at an address alike; a server from before these members is asked
with a chat request holding an empty image, which it refuses at once,
for one reason or the other. When the link does not, the box is red and
Send says why before anything goes:
| Case | What the window says |
|---|---|
| its own model has no projector | this model has no projector, it reads text only (a vision or audio model of the catalogue has one, or start the window with --mmproj FILE) |
| its own projector is of the other kind | this model's projector encodes images, not sounds |
| a server at an address without one | the server at URL has no projector (it was started without --mmproj), it takes text only |
| that server's is of the other kind | the projector of the server at URL encodes images, not sounds |
| an attached agent | an attached agent takes text and documents only |
A server that is not OKLM's has no such figures and is not understood
when it answers that request: the message is sent, and if the server
refuses it the status line shows its error and the images, sounds and
videos are left out of the next requests (their lines in the
conversation say not sent), so that the conversation can go
on in text. An image is sent again with each request, as the whole
conversation is; the server keeps its encoding.
A picture pasted from the clipboard. Ctrl+V with a
picture on the clipboard ("Copy image" in a browser, a selection copied
in an image editor, a screenshot tool) attaches it: the window asks the
clipboard's owner what it offers, and takes image/png,
image/jpeg, image/webp or
image/bmp, in that order. The picture is kept as
pasted-N.png in the window's temporary folder (the one of
the long documents' indexes, removed when the window closes) and joins
the row as a dropped file does: its box, its thumbnail,
Pasted: a picture from the clipboard (image/png), attached as pasted-1.png.
in the status line. A clipboard that offers text is pasted as text by
the field that has the keys, as before, also when it offers a picture
beside it (a spreadsheet's cells).
Files copied in a file manager. Ctrl+V after "Copy"
on one or several files attaches them, as dropping them does: the window
asks the clipboard for a list of files (text/uri-list, then
GNOME's x-special/gnome-copied-files) before anything else,
since a file manager offers the paths as text beside the list, and
attaches each file: address of this machine
(Pasted: 2 files copied in a file manager, attached.). An
address of the web or of another machine is passed; a clipboard whose
list names no local file is pasted as text. Run on 2026-11-15 under Xvfb
with tools/gui_copy_files.py owning the clipboard as a file
manager does: a PNG and a PDF copied, Ctrl+V, both in the row; then a
text copied, pasted in the message as text. On X11 it needs the same
clipboard module as a pasted picture. On Windows the list is the
system's own, CF_HDROP, which an Explorer window puts on
the clipboard for copied files: gui_clip_files() copies the
block out and att_hdrop_uris() (chat_attach.h) reads its
header and its names (UTF-16, or bytes from an older program) as the
same list of file: addresses, so the rest is what X11 runs;
a file of another machine's share (\\server\share\...) is
passed, as an address of another machine is. The reader works on bytes
and is checked on every system (test_hdrop in
tests/client/test_gui_attach.c: three names with a blank
and accents, a surrogate pair, byte names, an empty list, blocks cut
short); the Windows call is cross-compiled and has not been run.
It needs the clipboard module to carry pictures, which
nilorea-library gains with n_clipboard_get_data() (its
feature test, N_CLIPBOARD_HAVE_DATA, decides at build time:
without it Ctrl+V pastes text only). On X11 the picture is fetched
through the selection's targets, in pieces (INCR) when it is larger than
a request carries; on Windows it is the registered PNG
format, or the bitmap as a BMP file. Run on 2026-11-13 under Xvfb with
another program owning the clipboard: a PNG of 887 bytes pasted, sent to
SmolVLM-256M and described
(Red circle on blue background.), then one of 85 KiB
pasted. The Windows side was compiled, not run. The picture below is
red-ball.png of tests/fixtures/gui, copied as
a file manager copies a file (tools/gui_copy_files.py),
pasted, described, and pasted again for the next message.

Documents are read by the window, whether its server
is its own or at an address: the server receives text. A document goes
in the message before what was typed, under its name, in the form
oklm run --doc uses:
Document harbour.pdf:
<document>
...its text...
</document>
When does the ferry leave?
Its tokens are counted by the server
(POST /v1/messages/count_tokens), or taken as a third of
its bytes when the server does not count them
(about N tokens). Before sending, the documents' tokens are
compared with what the context leaves: its length (the one the side
panel shows; a server at an address says it at
GET /api/context), less the tokens of the conversation so
far (the last reply's prompt and its own), less the reply's share (the
reply length asked, a quarter of the context at most), less the typed
message and 48 tokens for the template. A context that is not known (a
server that is not OKLM's, an agent that does not tell it) is not
compared with. An attached agent tells its context in its state
(GET /agent/state: context, the model's
window, and context_used, what its instructions, its tools
and its conversation take of it), read when the window attaches and
again before each message with a document: the documents then have half
of what the window has left once that prompt and a quarter of it for the
reply are taken off, the other half being for what the agent reads while
it works.
A document that does not fit is searched. The window builds a retrieval index of it, kept between windows, and gives each message the passages found for that message, with the library's retrieval module (rag.md); the window does it all itself, so it is the same for its own server and for one at an address, which receives text:
Which documents. When the documents of a message do not fit together, the longest is searched, then the next, until those left whole take at most half of the room; the passages have the rest, shared equally between the documents searched.
The index. The document's text is cut into passages by the library's chunker (headings, paragraphs and sentences, about 256 tokens each; a source file by its definitions), each embedded, with the keyword postings beside the vectors.
The index is kept between windows, in
$XDG_CACHE_HOME/oklm/docs(else~/.cache/oklm/docs;%LOCALAPPDATA%\oklm\docson Windows), in a file named by the hash of the document's text, its length, and the hash of the embedding model's identity and width:dc0cb7138877a024-184033-e231dcf28a5cf44b.gguf. The same text attached again, in this window or another, under any file name, is found there: the index's builder sees its one source unchanged and embeds nothing, asoklm rag indexrun again does (rag.md), and the note saysIts index was found in the cache, built by an earlier window: nothing was embedded again. A text that changed by a byte, or another embedding model, is another file. The cache keeps 64 indexes and 512 MiB at most: those used longest ago go first.--no-doc-cache, or a cache folder that cannot be written, leaves the index a temporary file as before:doc-N.ggufin a folder of the window's own,oklm-chat-XXXXXXunderTMPDIR(/tmp), made for this user only, removed when the conversation is cleared or rewound past the document, and when the window closes.The embedding model is the one
--embed-modelnames (an installed model's name, a GGUF file, orhashfor the library's hashed bag of words, which needs no model and matches words only), else the first installed embedding model of the catalogue. It is loaded on the CPU at the first long document and kept until the window closes (bge-small: 130 MiB).The search is the index's hybrid one (vectors and BM25 fused, 48 candidates). The best passages are taken while they fit and written in the document's order, each under a line such as
[lines 807 to 817, under The observatory]. A message without a word gets the document's start. A passage is counted by the chunker's tokens, brought to the model's when the model counts more for the document, and its line of place by its length.A reranker, when one is at hand. The index finds the passages that resemble the message; a cross-encoder reads the message with each passage and says which answer it. When a reranking model is installed (the catalogue's
ms-marco-minilm-l6-v2: the first installed model of Retrieval whose line says it reranks), or--rerank-model NAME|FILEnames one, the 24 passages found first are read by it and taken in its order (oklm_rag_rerank_hits()); the note then saysput in order by the reranker NAME. It is loaded on the CPU at the first search and kept. Without one, with--no-rerank, or when it cannot be loaded (a line on the standard error says so), the index's order is kept. A message about the whole document is not reranked.What the model reads, in place of the whole text:
Document town.md (it is longer than the context: here are the 11 passages of its 100 found for this message, in the document's order): <document> [lines 807 to 817, under The observatory] ... </document> Who is the keeper of the observatory? Answer in one sentence.The messages after it each get their own passages, from every long document of the conversation (eight at most), and each shows the document's line with what it was sent. The passages of the message before are not sent again: a request carries the conversation, and the passages of the last message only. Ask about one thing at a time; the earlier answers stay.
Meanwhile the message has left its box, the status line counts the passages embedded, and Stop cancels: the message is back in its box and the cut is offered.
/saveand/exportwrite each message with the passages it was sent.
Measured on the CPU (six cores of a machine other work shared):
town.md, 120 KiB and 26872 tokens, was indexed in 41 s with
bge-small-en-v1.5 (100 passages) and in under a tenth of a second with
hash (105 passages); the search for a message took less
than a frame of the window. On 2026-11-15 (load average 1.0),
docs/rag.md, 180 KiB and 58 109 tokens for Qwen3 0.6B at a
context of 4096: indexed in 5.0 s with bge-small-en-v1.5 Q8_0 (295
passages) by a first window, a file of 612 576 bytes in the cache; found
there by the second window, nothing embedded. With
ms-marco-MiniLM-L6-v2 as the reranker and a reply of 80
tokens, 15 passages went with a question (a prompt of 3 787 tokens) and
15, spread over the document, with Summarize this document.
(3 610 tokens), both answered. That run showed a fault this entry fixes:
a passage's line of place was counted 12 tokens whatever it held, and
the first message, 19 passages under long headings, was refused by the
server
(the prompt holds 4365 tokens, the context 4096).
When a document cannot be searched (no embedding model is installed,
the model cannot be loaded, the search was stopped) the window says why
and offers Send the beginning, which cuts each document
in turn to what is left (in proportion to its count, at the end of a
line when one is near; the block then says
its beginning: N of M bytes, the rest did not fit), and
Cancel.
A message about the whole of a long document ("Summarize this.",
"What is this file about?", an outline, the main points,
tl;dr: English words, and résum) is not
searched for: the passages closest to those words tell little of the
document. It gets passages at equal steps over the document instead, its
first and its last among them, as many as the room takes, and
the model is told so: the block's first line then reads
Document town.md (it is longer than the context: here are 15 passages of its 295 taken at equal steps over the whole document, its first and its last among them, in the document's order, this message being about the whole; what lies between them is not shown):,
and the message's line in the conversation ends with
at equal steps over it.
An attached agent is sent passages too, once its
context is known: a document longer than what its context has left is
indexed (or found in the cache) and the message goes with the passages
found for it, under the same block. The agent keeps its own
conversation, so the window does not search the document again for the
messages after: attach it again with another question for other
passages, which the cache makes immediate. Run on 2026-11-15 with
oklm agent -m qwen3-4b -c 16384 --listen on OpenCL and
docs/rag.md (180 KiB, about 61 345 tokens): before its
model was loaded the agent's state told no context, the whole text went
(184 123 bytes) and the agent answered size overflow; with
its context told (16 384, of which 3 981 taken), 16 passages of the 295
went (14 871 bytes) and it answered from them:
When the passages found are judged irrelevant, corrective retrieval rewrites the question as a search query and searches again, potentially using web search if allowed. If still nothing relevant is found, it answers "No reliable source was found."
Limits. Those passages are a sample of the document, not all of it:
what lies between them is not sent, and a summary cannot tell it. The
passages are chosen by the retrieval index and, when there is one, a
reranker: no corrective step (no judge of the passages, no second
search), and the window does not show which passages went
(/export writes them). An agent whose state tells no
context (one from before the member, or one whose model is not loaded
yet) is sent the whole text. An image or a sound is 20 MiB at most (a
request carries 32 MiB), a document 64 MiB. Drops need Allegro 5.2.9 or
later on Linux (X11), where they are its events. On Windows Allegro has
none (its library there takes no drop): the window asks the system for
the files dropped on it (DragAcceptFiles,
WM_DROPFILES) and hands each on as that same event, so a
tool sees one kind (apps/gui/shell.c); text dragged from
another program is not taken there, files are.
Speech to text
A recording becomes text in the message box, where it is read and corrected before it is sent: speech recognition mishears names and numbers, and a message sent by itself cannot be taken back.
| Control | What |
|---|---|
| Record | starts a recording of the system's microphone; the button then reads
Stop, and a second press ends the recording and has it
transcribed. /listen does the same |
| the meter | in the place of the path's field while a recording is made: the time, and the level of the loudest fiftieth of a second of the last fragment on a decibel scale (empty at -60 dB of full scale, full at 0, amber above -12 dB). While the sound is transcribed, it says so |
| Enter | ends the recording, as Stop does (a command typed
in the message runs instead: /listen send) |
| Escape | while recording: cancels, nothing is kept. While transcribing: the text is dropped (a request to a server is cut) |
| Speech..., in the side panel | the Speech window: the microphone, the language spoken, the pause and the level that end a turn, and what is done by itself (the settings) |
| Transcribe | on the box of an attached recording: its words are written in the
message under Transcript of NAME:. When the model hears
nothing itself the box then leaves the row and the sound is not sent;
when it does hear (Ultravox), the file stays attached as well. On the
box of an attached video that has a sound track: the track's words, the
video staying attached (A video's
sound) |
Who transcribes depends on what the window is linked to:
| Link | Who | |
|---|---|---|
| the window's own server | that server, POST /v1/audio/transcriptions (server.md) |
started with the Whisper model --transcribe-model names
(a directory, or an installed model's name), else with the smallest
installed speech recognizer of the catalogue;
--no-transcribe for none. A model installed while the
window runs is loaded by the window itself at the first use |
| a server at an address | that server, the same route | when its figures say "transcribe":true
(GET /api/stats; it was started with
oklm serve --transcribe-model). Otherwise nothing is
transcribed and the window says
the server at URL has no transcription model (it was started without --transcribe-model):
a sound meant for that server is not given to another |
| an agent | the window itself, with the library (oklm_whisper) |
an agent's link carries text only. The model is loaded at the first use and kept; the text then goes to the agent as any message |
The side panel says which under the figures
(Speech to text: whisper-base, or
the server's), and the Record button's
tooltip says why when nothing transcribes.
The window's server checks its Whisper model's files when it starts
and loads the weights when a sound is first transcribed, as
oklm serve --transcribe-model does (server.md): a
window in which nobody records does not pay for them. With Whisper base,
measured on the CPU with SmolLM2-135M as the chat model when the server
loaded the model at its start: 253 MiB for the weights and 283 MiB more
once a sound was transcribed; the load itself takes two tenths of a
second, within the first transcription. --no-transcribe
avoids both. The 11-second sample (jfk.wav) is transcribed
in 2 to 8 seconds on a 16-thread CPU that other work shares; three
seconds of speech take nearly as long, since the recognizer works on
windows of 30 seconds.
What is sent. A recording goes as a 16-bit PCM WAV
file at 16 kHz mono, the body of the request, with
response_format=verbose_json (the answer's
duration is the one the status line shows) and
language when one is chosen. An attached file goes as its
own bytes: the server decodes what the window decodes.
The language. The Speech window's
Language chooser lists
Found by the recognizer (auto), then the languages the
transcriber knows: a server's from its figures (languages
of transcribe_stats), the window's own Whisper model's from
its tokenizer (oklm_whisper_describe()), English alone for
a model of English only.
- Found by the recognizer: nothing is sent, and the
recognizer finds the language in each recording (the window's own does
through
oklm_whisper_run()). The status line names it and how sure the model was:Transcribed 4.2 s of sound in 0.9 s (by the window's server, with whisper-base, language found: English (en), 100 % sure). A short or noisy recording may be taken for another language. - A language: it is given with each transcription
(
language=de), and the line saysin German (de). An English sentence asked for as German came back as German words, which is what a wrong choice does. - The choice is kept for the next window, in
gui-chat-languageof the settings folder ($XDG_CONFIG_HOME/oklm), so that someone who dictates in French is not found to speak another language by a short recording.--transcribe-language CODE(orauto) at start goes before what was kept, and is kept only once the chooser is used. - Written in English: with Write what is said
in English (translation) under the chooser, or
--transcribe-translate, what is said in another language comes back in English: Whisper's translation task, on the window's own model, orPOST /v1/audio/translationsof the server in place of the transcriptions route. The line ends withwritten in English. A model of English only (whisper-base.en) has no such task: the window's says so, and a server answers with its refusal. The box is not kept for the next window.make gui-testruns it on the tiny Whisper fixture (language found: French (fr), 75 % sure, written in English); with a person speaking another language it was not run.
The words while speaking. While a recording is made
(Record, /listen, a turn of a spoken conversation) the status line
reads
Heard so far: And so my fellow <Americans, ask not.>:
what the recognizer heard of the recording so far. Whisper reads a whole
sound, so this is the recording so far transcribed again, every two
seconds of new sound (the first after one second; twice what the last
answer took when that is more), one at a time, on a thread of its own:
by the server with partial=true, which refuses at once when
its recognizer is at work (server.md), or by the
window's own recognizer when no whole recording holds it. The words two
readings in a row start with alike are shown as said; what the last one
heard after them stands between angle marks and the next reading
replaces it. Past twenty seconds of sound the segments said are kept and
the next readings start after them, so a long recording costs its last
sentences each time, not its whole length (listen_partial
in apps/gui/chat_listen.h). The message never gets that
text: at the recording's end the whole recording is transcribed once, as
before, and its text is the one written. A server at an address is asked
only when its figures say it gives partial texts
(transcribe_stats.partial). Run on 2026-11-15 with Whisper
base on the window's own server and --mic-file jfk.wav:
after 1, 3, 5, 7 and 9 s of sound, | And so many!,
And so | my fellow Americans.,
And so my fellow | Americans, ask not.,
And so my fellow | Americans ask not what your country can do for,
And so my fellow Americans ask not what your country can do for | you, ask what you
(said before the bar, heard after it), each in 0.3 to 0.7 s, then the
message:
And so my fellow Americans, ask not what your country can do for you, ask what you can do for your country.
The microphone is Allegro's audio recorder, asked
for 16 kHz mono 16-bit; a device that gives another rate or two channels
is mixed to mono and resampled once the recording ends (the library's
windowed sinc). Its fragments, a twentieth of a second each, arrive as
events of the window. A recording is kept in memory
only: nothing is written to disk. It ends by itself at the
longest sound the transcriber takes: what the server's largest upload
holds (upload_max of its figures: 17 minutes for the
default 32 MiB), 30 minutes at most.
The settings are controls of the Speech window (Speech... in the side panel), each started by an option of the command line. They apply to the next recording and the next transcription; a spoken conversation under way takes the level and the last box at once, the pause when it is next started.

| Control | Option | Default | Why |
|---|---|---|---|
| Microphone | --mic NAME |
the system's default input | the capture devices found, by the names PulseAudio or PipeWire gives
them (pactl list short sources; the outputs' monitors are
left out), looked for again every five seconds while the window is
open |
| Language | --transcribe-language CODE|auto |
the one chosen last, else found by the recognizer | above |
| Write what is said in English | --transcribe-translate |
off: what is said is written as it is said | Whisper's translation task, or the server's translations route (above) |
| Pause that ends a turn | --listen-silence SECONDS |
1.2 s | 0.3 to 5 s: the pause that ends a turn of a spoken conversation. Shorter, a breath sends half a sentence; longer, the reply waits |
| That pause ends a recording too | --listen-silence SECONDS |
off: a recording ends on a press | a pause to think would end it. With it, a recording made with
Record or /listen ends once something was
said and nothing since for that long |
| Level that is speech | --listen-threshold DB |
-40 dB of full scale | -60 to -5: the level above which the microphone hears speech, for that pause and for a spoken conversation. Raise it in a noisy room |
| Send what was said once it is transcribed | --listen-send |
off: the text waits in the message | a misheard word is corrected before the model answers it;
/listen send asks for one recording |
| Transcribe an attached recording without a press | --transcribe-attached |
off: it waits for Transcribe | transcribing takes the recognizer for a time that grows with the recording and writes in the message being typed: a drop does not start that by itself |
| In a spoken conversation, speaking over the voice stops it | --no-interrupt (off) |
on | off for loudspeakers, which the microphone hears |
| Show the words while a recording is still coming | --no-partial (off) |
on | the partial texts take the recognizer's time while you speak: a tenth to a quarter of it with Whisper base or on a device, half of it with a large model on the CPU, where they come too late to be of use (the words while speaking) |
Choosing a microphone. Allegro's recorder opens the
capture device the sound system gives it, and PulseAudio's client
library (PipeWire's too) records the source that the
PULSE_SOURCE variable names when it starts. A choice
therefore sets that variable and stops the window's sound system, with
the stream that plays the voice; the next recording or reading starts it
again. It is refused while something is recorded or said. Run on
2026-11-13 on a machine whose PipeWire has one input (a USB codec): the
chooser listed it under the system's default, the status line said
Microphone: alsa_input.usb-..., from the next recording on.,
and Record then recorded three seconds from it
(silence: nothing is plugged in that input). On Windows, and on Linux
without pactl, the list is the system's default alone; with
ALSA alone the choice has no effect.
What the window says when it cannot:
| Case | What it says |
|---|---|
| no Whisper model installed | no Whisper model is installed: Models has whisper-base (281 MiB) under the sound models, or name one with --transcribe-model |
| no sound device at all | No microphone: this machine has no sound device the window can open. An attached recording can still be transcribed. |
| a sound device without a capture device | No microphone: the system gives no capture device (none is plugged in, or another program holds it). |
| a recording of silence | Only silence came in the 3.0 s recorded: nothing was transcribed. Is the microphone the system's input, and not muted?
(nothing above -55 dB: a recognizer makes words up from silence) |
| a recording under a quarter of a second | The recording is too short to hold a word |
| the transcription fails | the reason, and the recording joins the row of files as
recording-1.wav, still in memory only:
Transcribe on it tries again, its x drops
it |
--mic-file FILE is a test aid: the window records that
sound file in place of the microphone, in real time, and the file's end
is the second press (given several times, it scripts a spoken conversation;
Record takes the first file). A WAV file is fed at its
own rate, so that one at 44.1 kHz takes the path of a device that gives
no 16 kHz. With it, and with --screenshot, each
transcription is also printed on the standard error
(oklm-gui: transcribed 11.0 s in 3.3 s by ...: TEXT). It is
how the recording is checked on a machine without a microphone.
Limits. The words appear once the recording ended, not while
speaking. The language found is the recognizer's for the whole
recording; the route's translation into English
(POST /v1/audio/translations) has no switch in the window.
A microphone is chosen through PulseAudio or PipeWire only, and the
choice was run on a machine with one input, without a voice in front of
it. The recorder was run on Linux (PulseAudio or PipeWire, ALSA). The
Windows build's was run under Wine 11 on 2026-11-13, on a machine
without a sound input there: Record opened Allegro's
recorder, counted the seconds, and the stop said
Only silence came in the 3.7 s recorded: nothing was transcribed. Is the microphone the system's input, and not muted?;
with --mic-file the sound file was transcribed by the tiny
Whisper fixture as on Linux; without a Whisper model the line names the
one to install. A voice recorded by a Windows sound card was not.
Replies read aloud
A reply, or any text, is made into speech and played while the
conversation goes on. The speech model is OuteTTS 0.2 or 0.3 (English,
one built-in voice and those there are speaker files for, slower than it
is heard on a CPU) or Soprano (soprano-1.1-80m: English,
one fixed voice, faster than it is heard on a CPU), both on the CPU (speech.md).
The sound is played at the rate of who speaks: 24
kHz for OuteTTS, 32 kHz for Soprano. The window reads it in the headers
of its own speech model's files before anything is loaded
(oklm_speech_probe()), and for a server in its figures
(sample_rate of speak_stats; each answer's
X-Sample-Rate is checked against it). What plays is opened
at that rate, and opened again when who speaks changes to a model of
another rate; the sounds kept with the replies are then made anew. Run
with Soprano on six shared cores, the voice written to a file:
spoke 1 part, 3.6 s of sound in 0.5 s by the window's server, with soprano-1.1-80m, speed 1.20,
and by the window itself beside a server without a voice
4.4 s of sound in 0.5 s; both files are 32 kHz, and Whisper
reads the sentence in them.
| Control | What |
|---|---|
| Speak, beside the name over a reply | reads that reply; while it is read it says Stop, in amber, and a press ends the voice. A reply that was read to its end is heard again from its kept sound, at once and without a new synthesis (below). It shows on every reply of the model that has a text and is not being written; dimmed where nothing reads aloud, its tooltip saying why. A press on it selects nothing: dragging and double-clicking the text work as before |
| Read aloud, beside Record | every new reply is read while it is written, sentence by sentence.
Off when the window opens (--read-aloud: on). A reply cut
by Stop while it is written is said up to its last
whole sentence. An attached agent's text is read when its turn ends |
| Voice:, beside Talk | the voice: one of those the server that speaks offers, or, when the window speaks itself, the model's built-in one or a speaker file (below) |
| Pause, Resume | at the row's end while something is read, before Stop
reading: the voice stops where it is, and goes on from there.
What is made meanwhile waits: nothing is lost and nothing is asked
again. /pause does the one or the other |
| Speech..., in the side panel | the Speech window: Speed of the voice, 0.25 to 4 times its own pace with the pitch kept, and Play a reading once its whole sound is made. Both apply from the next reading |
| the row's meter | in the place of the path's field while something is read:
Loading the voice,
Part 1 of 3: the first sound is made,
Speaking, part 2 of 3, Speaking, the end; for
a reply being written,
The voice waits for the reply's first sentence, then
Speaking while the reply is written, sentence 2; on its
right the time since the press and the seconds of sound made so far |
| Stop reading | at the row's end while something is read: what is queued is dropped and the request behind it is closed. It ends a paused voice too |
| Escape | stops the voice, when no recording is made and no list of commands shows |
/speak [on|off|stop|pause|resume|TEXT] |
alone, the last reply; on and off, what
Read aloud does; stop, what Stop
reading does; pause and resume, what
Pause and Resume do; anything else is
a text to read. The five words count when they are the whole argument,
in any case |
One thing is said at a time: a second Speak, or
/speak, stops the first, and so does a new reply with
Read aloud on. /reset and New
chat stop the voice; /retry and
/rewind do when they remove the reply being read. Linking
the window to something else stops it too.
What is read is what the dashboard's chat reads (server.md), by the same
rule (speak_sayable() in
apps/gui/chat_speak.h, checked on the page's own table of
texts): not the reasoning (a leading <think> block),
a fenced code block as "Code omitted.", a link as its text, a bare
address as "link", an image as its description, no mark of Markdown; a
heading, a list item, a table's row and a paragraph each end as a
sentence. A text longer than a server reads in one request
(max_input of its speak_stats, 4096 bytes by
default) is cut at the ends of sentences and its pieces are asked for
one after the other: they are the parts the row counts. A piece without
a letter or a digit is left out, and a reply with nothing to say (code
only, reasoning only) answers
Nothing in it can be read aloud.
While it is written. With Read
aloud on (and in a spoken
conversation) a reply does not wait for its end. Each time the
window draws, what was written since is read, and every sentence that is
certain goes to the voice, as a piece of its own in a queue the thread
making the sound reads: a request a sentence to a server, a call a
sentence to the library, in order. A sentence is certain once a blank
follows its ., ! or ? (the point
of 3.14 ends nothing) and nothing written later can change
what it says:
| What is being written | What the voice gets |
|---|---|
| a paragraph | each sentence as it ends; the last one at the reply's end |
the reasoning (a leading <think> block) |
nothing; the answer starts right after
</think> |
| a block of code | "Code omitted." when the fence opens, nothing of its lines |
an emphasis, a code span, a link or a tag still open
(**the short. waves) |
nothing until it closes: the sentence could be the link's address, or lose its marks |
a line that holds a character that may still be a mark
(snake_case, 2 * 3, a [, a
<, a table's row) |
the line once it ended, not sentence by sentence |
| a heading, a list item | the line once it ended |
The sentences given this way are the ones the whole reply gives
(speak_stream in apps/gui/chat_speak.h; the
test feeds each text of its table whole, byte by byte and cut in two at
every place, and 2000 random runs of words and marks the same way). One
case cannot be known in advance: a model that closes a reasoning it
never opened (</think> alone). What was said until
then was the reasoning: the voice is stopped and starts again with the
answer.
The voice is started when the message is sent, on an empty queue, so
that the window's own speech model is read from disk while the first
sentence is written; ticking Read aloud or
Talk reads it at once. And the first piece of a reading
asks the synthesizer for a first pass of five words
(first_words, speech.md): a
pass is heard only once it is whole, so the first sound comes after five
words' worth of work, not twelve; the rest of the piece follows under
the same prompt, in the same request. (The window used to cut the piece
itself and ask twice, which cost the voice's prompt a second time.)
Measured on the CPU (16 threads, shared with other work: the load
average was 16 to 24), Qwen2.5-0.5B asked
Tell me three facts about the sea, one short sentence each.,
the window speaking itself with outetts-0.2-500m, a reply
read after its end (as the window did before) and one read while it is
written run in turn, five times each:
| Chat model, who speaks | Read after its end: the first sound, seconds after the message was sent | Read while it is written |
|---|---|---|
| Qwen2.5-0.5B, the window itself | 19.6 to 34.8, median 22.1 | 10.4 to 17.4, median 13.9 |
Qwen2.5-0.5B, the window's server (--speak-model) |
11.8 to 32.3, median 28.2 | 7.4 to 16.8, median 15.2 |
| SmolLM2-135M, the window itself | 13.1 to 21.3, median 18.2 | 8.0 to 12.1, median 8.7 |
The replies ended 0.5 to 7.6 seconds after the message was sent, their first sentence 0.2 to 2.2 seconds after: the rest of the wait is the voice's, 0.07 to 0.2 second of sound per second under that load, its first pass taking the voice's prompt (about 890 tokens) before the first word. The gain is the reply's remaining sentences, which are no longer waited for, and the first pass's seven words fewer.
The voice. The Voice: chooser lists the voices of who speaks, read again every two seconds, and a voice applies from the next reading:
| Who speaks | What the chooser lists | What choosing a voice does |
|---|---|---|
| a server at an address | that server's voices, by name: the one it was started with first,
then the others of GET /v1/audio/voices (server.md), read in its figures
(speak_stats.voices) |
each request names it ("voice": "anna"); the first row
asks for none, the server's own. --speak-voice NAME chooses
one at start |
the window's own server (--speak-model) |
the same: the voice it was started with
(--speak-voice FILE, else the built-in one), the built-in
one, and the speaker files of the user's voices folder and of the speech
model's folder, which the window gives its server |
each request names it. The server keeps its models and takes the other speaker file: nothing is started again, and the conversation is untouched |
| the window itself | the speech model's built-in voice and the speaker files it finds in
the user's voices folder and beside the speech model's file; a file
named by --speak-voice is in the list and chosen |
its synthesizer takes the speaker file before the next text
(oklm_speech_set_voice()), keeping its models loaded |
Soprano has one voice. A server that speaks with it lists
soprano alone; when the window speaks itself with it the
chooser is dimmed, its tooltip saying that the model takes no speaker
file, and none is given to it.
The user's voices folder is ~/.config/oklm/voices/
($XDG_CONFIG_HOME/oklm/voices,
%APPDATA%\oklm\voices on Windows). A speaker file is
OuteTTS's JSON form, the words of a recording with their durations and
audio codes (speech.md). Run against
oklm serve on the test's small models with three voices
(tiny-voice, the one it was started with,
en_male_1 and anna, a file of its
--speak-voices folder):
$ oklm-gui --url http://127.0.0.1:7457 --speak-voice anna --speak-speed 1.25 --speak-file said.wav --send "/speak Hello there, world. One two three."
oklm-gui: spoke 1 part, 0.2 s of sound in 0.1 s by the server at http://127.0.0.1:7457, voice anna, speed 1.25
$ oklm-gui --url http://127.0.0.1:7457 --speak-voice nobody ...
oklm-gui: not spoken: 'voice' must be "default" or one of the voices this server offers: "tiny-voice", "en_male_1", "anna" (...)
A voice that who speaks refuses (a name the server does not offer, a
file that is no speaker file, codes the model does not know) leaves the
voice as it was, with the reason in the status line. OKLM does not make
a speaker file: a voice is cloned from a recording with WavTokenizer's
encoder and an aligner, which it has not (speech.md); the outetts
Python package writes one (interface.save_speaker()).
The speed. Speed of the voice, in
the Speech window (--speak-speed X at start), goes from
0.25 to 4: a server is asked for it ("speed": 1.25), the
window's own synthesizer is given it, and both change the pace with the
pitch kept (speech.md). At 1.3 the
sentence of the runs below is 3.6 s of sound, for 4.7 s at 1.
Pause. Pause stops the voice where
it is: the playing stream is stopped with what it holds, and what it was
not given yet waits. The synthesis is not stopped: the rest of the sound
goes on being made, and waits too. Resume goes on from
the same sample, with nothing asked again. Run with the voice written to
a file: the reading paused a second after it was asked, the whole
sentence made meanwhile
(the voice goes on: 4.7 s of sound waited), and the file,
read by Whisper, holds
The quick brown fox jumps over the lazy dog, then it sleeps in the warm sun.
once and whole.
A reply's sound is kept. Every sample made for a
reading is kept until the reading ends, and the sound of a reply read
from its first word to its last stays with the reply, in memory.
Speak on that reply then plays the same samples at
once: no request, no synthesis, and the status line says
Heard again: the 16.1 s of sound of its last reading were kept, nothing is made anew.
It is made anew when the voice, the speed or who speaks changed since,
and when the reading was stopped or failed before its end. The replies'
sounds take 48 KiB a second and hold in 64 MiB together (22 minutes):
beyond, the oldest are dropped. /retry,
/rewind and New chat drop them with their
replies; a text read by /speak TEXT is not kept.
Played once whole. On a CPU the voice is made slower
than it is heard, so a reading comes in bursts of a few words. With
Play a reading once its whole sound is made
(--speak-whole) nothing is played until the last pass is
made, then all of it, without a wait between its words; the row says
Making the sound, part 2 of 3: played once whole meanwhile.
The sentence above, made in 46.8 s on six shared cores, was then played
in its 3.6 s. It is a trade: the first word comes with the last. A reply
read while it is written waits for its end too.
Who speaks depends on what the window is linked to:
| Link | Who | |
|---|---|---|
the window's own server, started with
--speak-model NAME|FILE |
that server, POST /v1/audio/speech (server.md) |
the speech model is loaded with the chat model when the window
starts. A catalogue name finds its vocoder; a GGUF file takes
--speak-vocoder FILE. --speak-voice FILE is
the voice chosen at start |
| the window's own server, without the option | the window itself, with the library (oklm_speech) |
the first installed speech model of the catalogue, loaded at the first reading and kept until the window closes. One installed while the window runs is found within two seconds |
| a server at an address | that server, the same route | when its figures say "speak":true
(GET /api/stats; it was started with
oklm serve --speak-model) |
| a server at an address that has no voice | the window itself | the text of a reply is the window's to read: it is given to no other server |
| an agent, a session being read | the window itself | an agent's link carries text only. With Read aloud, the agent's last text is read when its turn ends, not its tools' calls and results |
--no-speak: nobody. The side panel says which under the
figures (Read aloud: off, outetts-0.2-500m, or
the server's voice).
The speech model is not loaded by default. It takes
1.6 GiB and its voice is wanted by few conversations, so a window
started without --speak-model loads it only when something
is first read, on the thread that makes the sound: the window keeps
drawing, and the row says Loading the voice. Measured on
the CPU with SmolLM2-135M as the chat model and Whisper base beside it:
1005 MiB without the speech model, 2627 MiB once the window read a text
itself; with --speak-model, the window's server holds the
model and loads it at the first reading too (half a second, then 1.6 GiB
more: 2616 MiB after a reading). A window on a larger chat model has to
have that much free.
What is asked. By a server:
{"input": PIECE, "response_format": "pcm", "words": 12},
with "first_words": 5 for the first piece of a reading,
"speed" when it is not 1 and "voice" when one
was chosen: raw 16-bit samples at 24 kHz mono, sent pass by pass while
they are made, twelve words a pass at most so that the first sound comes
early. The window reads the answer as it arrives
(oklm_client_post_data()), a sample cut between two pieces
put together. By the window itself:
oklm_speech_synthesize_chunks() with the same twelve words,
the same first pass and the same speed, each pass's samples given when
the pass is done. A stop closes the connection, which ends the server's
synthesis (its speak_stats.cancelled goes up by one), or
sets the flag the library reads between tokens.
The samples of a reading join its sound (speak_take in
apps/gui/chat_speak.h), which gives them to what plays as
they come, or holds them while the voice is paused or the reading is
wanted whole.
Playback is a stream of Allegro's audio (16-bit, mono, 24 kHz, four fragments of a twentieth of a second, mixed to the device's rate by Allegro's mixer) that a thread of its own feeds from a queue: the thread that makes the sound adds to the queue and never waits for the device, and the window's thread touches neither. A pass plays without a gap, since its samples are in the queue whole. Between two passes the stream is given silence when the next one is not made yet, which is the case on a CPU that makes less than a second of sound per second: the voice then says a few words, waits, and goes on. The stream is opened at the first reading, paused once it had only silence for a second, and closed with the window. A stop empties the queue and mutes the stream while the fifth of a second it still holds goes by.
What the window says when it cannot:
| Case | What it says |
|---|---|
| no speech model installed, own server or agent | Nothing here reads aloud: no speech model is installed: Models has outetts-0.2-500m (636 MiB) under speech synthesis, or name one with --speak-model. |
| a server without a voice, and none installed here | Nothing here reads aloud: the server at URL has no speech model (it was started without --speak-model) and none is installed on this machine |
| no sound device | No sound can be played: this machine has no sound device the window can open. Nothing was asked of the speech model.
(or the system gives no playback device): the device is
opened before any sound is asked for, so nothing is synthesized for
nobody to hear. The tooltips of Read aloud and
Speak then say so; the next press tries again |
| the server's voice is busy (four texts wait already, 503) | Not read aloud: the server's voice is busy with other texts: ask again in a moment. |
| a reply of code or reasoning only | Nothing in it can be read aloud: the voice reads English words and numbers, not the code nor the reasoning. |
| the speech model cannot be loaded | Not read aloud: the speech model NAME cannot be loaded:
and the reason |
--speak-model NAME for a catalogue model that is not
installed |
the window does not start: oklm models pull NAME |
--speak-model FILE without --speak-vocoder
for a file that is not the catalogue's |
the window does not start, and says so |
--speak-file FILE.wav is a test aid, as
--mic-file is for the microphone: the samples that would be
played are written to that WAV file, one text after the other, and
nothing is played; no sound device is opened. With it, and with
--screenshot, each reading is also printed on the standard
error
(oklm-gui: spoke 2 parts, 8.6 s of sound in 39.1 s by ...,
oklm-gui: the voice was stopped after ...,
oklm-gui: not spoken: ...), and --screenshot
waits for the end of a reading. It is how reading aloud is checked on a
machine that plays nothing: oklm transcribe then tells what
the file says.
Limits. English, and the voices there are speaker files for: the
window makes none. On a CPU the voice is made slower than it is heard
(0.1 to 0.45 second of sound per second in the runs of this page, on a
16-thread CPU that other work shared): a reply is heard in bursts of a
few words with silences between them, and the first words come some
seconds after the first sentence, unless the reading is played once
whole. A reply's sound is kept in memory only, for the window's life,
and not written with /save or /export; the
route's flac is not asked for. The pause, the speed, the
kept sound and the whole reading were run with the voice written to a
file: nobody heard them from a loudspeaker. A pause holds the device's
stream as Allegro stops it, which was not run on a sound device. Reading
while a reply is written makes more passes than reading it after (a
sentence a piece), and each pass starts with the voice's own prompt: the
whole reply is said later than it would be in fewer passes. An attached
agent's text is read when its turn ends. The playing stream was run on
Linux, once, on the session's sound server: a sentence of 2.2 seconds
was made, queued and played out, with nobody in front of the loudspeaker
to hear it; without a sound server and without an ALSA configuration the
window said that no device opens. The Windows build has it compiled, not
run.
A spoken conversation
Talk leaves the microphone open and answers by voice: what is said is sent at a pause, without a press, the reply is spoken while it is written, and the window listens again when the voice has finished. It is speech to text and replies read aloud with nothing to press between them, and it needs both: a link for which something transcribes and something speaks (the window's own server or a server at an address; the side panel says who). Attached to an agent it answers that it is a conversation with a model: an agent's text comes when its turn ends.
| Control | What |
|---|---|
| Talk, beside Read aloud | starts the mode, or leaves it. --talk starts with it
on |
| the mode's row | in the place of the path's field: a dot of the state's colour, the
state, the time since it started, the microphone's level while it is
listened to, Escape leaves |
| Escape | leaves the mode, whatever it was doing |
| Stop reading | stops the voice: the window listens again |
/talk [on|off] |
alone, starts the mode or leaves it; on,
off |
| State | What the window does | What ends it |
|---|---|---|
| Listening (green) | records the microphone. Before anybody speaks only the last half
second is kept; Listening: you speak once the level
held |
the pause: what was recorded is the turn |
| Transcribing (amber) | has the turn's words found, by who transcribes for the link | the words: they are sent. No word: back to listening, nothing sent |
| Thinking (blue) | the message went, the reply is being written; the voice works on its
first sentence. Thinking; the voice is loading while the
window reads its own speech model (ticking Talk starts
that) |
the first sound |
| Speaking (violet) | the reply is said while it is written, then to its end | the voice's end, or somebody speaking |
A turn starts when the microphone's level stays
above a threshold for a quarter of a second, the dips shorter than a
fifth of a second (between two syllables) not counting, and ends when it
stayed under it for the pause. The level is the root mean square of each
fiftieth of a second (listen_turn in
apps/gui/chat_listen.h):
| Setting | Default | |
|---|---|---|
--listen-threshold DB |
-40 dB of full scale | what is speech. A click, a key or a door is louder and shorter than the quarter second: it starts nothing. Noise under the threshold never does |
--listen-silence SECONDS |
1.2 s | the pause that ends a turn. Shorter, a breath sends half a sentence; longer, the reply waits |
Nothing is sent for a turn that holds only silence or noise: one the level never held in is not a turn; one shorter than a quarter of a second or with nothing above -55 dB is not transcribed (a recognizer makes words up from silence); one in which Whisper heard no word is dropped. If you go on speaking while the words of a turn are being found (the pause was a breath), they wait in the message box and are sent with what follows, as one message. A turn is as long as a transcription takes: 17 minutes for a server's default upload.
A message typed and sent while the mode listens is answered as a spoken one: its reply is said, and can be spoken over.
The words are sent as they were heard: a spoken conversation has no moment to correct them. A name or a number that Whisper mishears is answered as it was heard; Record is the way to read before sending.
Speaking over the reply. While the reply is thought of and spoken the microphone stays open, and the same rule listens: when the level holds above the threshold for a quarter of a second, the voice stops at once (what is queued is dropped, the request to a server is closed), the reply stops being written (the request is cancelled; what was written stays in the conversation), and what you are saying is the next turn, from half a second before the level held.
The window cannot tell your voice from its own: there is no echo
cancellation. With loudspeakers the voice comes back into the
microphone and stops itself after a quarter of a second. Use
headphones; or start with --no-interrupt (the microphone is
not listened to while a reply is thought of and spoken; Stop
reading and Escape still stop the voice); or raise
--listen-threshold above the level the loudspeakers reach
at the microphone, which the mode's row shows.
What to expect on a CPU. The mode is as fast as its
three models. The first turn also pays for reading them: the window's
server loads its Whisper model at the first transcription (0.2 s for
Whisper base here) and, started with --speak-model, its
speech model at the first reading (0.5 s), the row saying
Transcribing what was said, then Thinking,
meanwhile; the window's own speech model is read when
Talk is ticked. On the 16-thread CPU of these runs,
shared with other work: a question of four seconds was transcribed by
Whisper base in 2 to 3 seconds; Qwen2.5-0.5B wrote its first sentence
within 2 seconds; and the voice, which makes 0.1 to 0.2 second of sound
per second there, was first heard 6 to 17 seconds after the message
went, then in bursts of a few words. A conversation at the pace of
speech needs a speech model several times faster than this one on a
CPU.
--mic-file scripts the mode on a machine without a
microphone. Given several times, each sound file is fed as the
microphone at its time, with silence between them: FILE or
FILE@SECONDS counts from the moment the mode starts,
FILE@voice+SECONDS from the first sound of the reply that
follows the recording before it, which is how a script speaks over a
reply whatever the machine's speed. With --speak-file the
voice goes to a WAV file, and each step is printed on the standard error
(oklm-gui: talk 4.4 s: transcribing); with
--screenshot the window is saved two seconds after the last
recording was answered:
oklm-gui -m qwen2.5-0.5b --backend cpu --talk --mic-file question.wav --mic-file other.wav@voice+3 \
--speak-file said.wav --screenshot out.png
Limits. No echo cancellation (above). The words of a turn show once it ended, not while speaking. The pause and the level are controls of the Speech window (the settings); the language is the one its chooser has, found in each turn when none is chosen, and the voice is English. An attached agent is not talked with. The device path was run with the test source and the file sink only: the test machine's capture device gives silence, and nobody was in front of a microphone.
The side panel
| Control | What |
|---|---|
| Server | the server's figures, read every two seconds from
/api/stats: requests generating over the sequences,
requests waiting, tokens a second, tokens generated, prompt tokens
evaluated and taken from the cache, requests answered, the context
length, TLS and keys. Under them, what the commands set: the start of
the system prompt (/system), the sampling values given
(/top-k, /top-p, /min-p,
/seed), the reasoning switch (/thinking), the
retrieval index with whether it is in use and what reorders its passages
when the window's server was given a reranker
(Index: docs.gguf (in use), reranked by a cross-encoder),
who transcribes and who reads aloud (with whether Read
aloud is on), the voice when it is not the built-in one or the
server's own, its speed when it is not 1 and whether a reading waits to
be whole, and whether Talk is on |
| Temperature | 0 to 2 (0.7), for the next message: 0 always picks the likeliest
token. /temp moves it, beyond 2 too. It is the
Temperature of the model
settings: each follows the other |
| Reply length (tokens) | 16 to 4096 (1024): max_tokens of the next message.
/max-tokens moves it, beyond its ends too. The model
settings' Reply length |
| Context (tokens), Apply context | the context size |
| Models... | the Models window: below |
| Speech... | the Speech window: the voice's speed and a reading played once it is whole (Replies read aloud), the microphone, the language and the settings of the recordings (Speech to text) |
| Connect... | the Connect window: Connecting |
| The model's name, a button at the bottom | the model the window talks to; a press opens the model settings |
| The line under it | the address |
The Models window
Models... (or --models at start) opens
the catalogue of models.md inside the chat
window. It is the model manager with
one more button: the list, the selected model's description, the licence
box, the download's bar, Download and
Delete are the same and are described there.
- Use this model serves the selected model. What it
is loaded with is the window's settings less what belongs to the model
before, then what is kept for the model itself (which wins), and the memory line judges those settings before
anything is stopped. When they are the settings the window runs with, a
window on its own server has the model loaded in place
(
oklm_server_switch(), server.md): the server waits up to thirty seconds for a reply being written, frees the model and its cache, and loads the other with the context, the backend and the sound models it has; the window goes on meanwhile, nothing is sent, and a note or the status line then saysThe window now serves NAME, loaded in 2.8 s, with a context of 4096 tokens. The conversation is kept; the model reads it again on the next message.A model that cannot be loaded or served leaves the reason and the model before still serving. When they differ, the model is loaded once, with them: the window's server is stopped and one is made of the other model with its backend, its layers, its context and its cache, with no question (the switch is the decision). The note then says which it was:- settings kept for the model:
loaded once in 2.8 s with the settings kept for it (context 8192, --gpu-layers 20 -ctk q8_0); - nothing kept, and something set for the model before, in the panel
or on the command line:
loaded in 0.5 s with the window's settings (context 4096). What was set for the model before is automatic again: --gpu-layers., and, for a model trained with less than the window's context,The context was reduced from 8192 to 2048 tokens, the most this model was trained with.; - kept settings the memory line refuses, or that do not hold once
loaded: they are left aside, kept as they are, and the model is loaded
with the window's
(
with the window's settings ...: the settings kept for it were left aside, with a context of 131072 tokens it does not fit: what is left for the cache holds 0.7 contexts, less than one; a context of 131072 tokens takes 18.0 GiB., for a model kept with every layer on the device at that context); - settings that do not fit, the window's alone included: the model is
not loaded and nothing is stopped
(
NAME is not loaded: with a context of N tokens it does not fit: ... The model before stays., with the line's reason). A model its device holds with less than one context of cache beside it is not such a case when the layers are left to what fits: the memory line places fewer of them there, and the model is loaded so. A window with no server of its own (started without a model, linked to a server, or reading a session) starts again with-m NAMEand the options it was given, less the scripted ones (--send,--settings,--press,--screenshot). It is offered for the text models (chat, coding, security) and for the vision and audio models (SmolVLM, Ultravox), which the window serves with the projector the catalogue has for them: images, or sounds, then go with a message./model NAMEdoes the same from the message box.
- settings kept for the model:
- For a model that is not a chat model the button is named after what
it is for, and the description says it. An embedding or a reranking
model (the Retrieval group): Open in
Retrieval starts the retrieval manager
(
oklm-gui --tool rag --model NAME), in a window of its own, on its Build tab with that model chosen as the one an index is built with (a reranker is used by its Search tab by itself). Whisper: Transcribes sounds: how writes where it is used,oklm transcribe -m NAME FILE.wav, oroklm serve --transcribe-model NAMEfor a server's clients. A speech-synthesis model (OuteTTS): Read replies aloud with it turns Read aloud on, and says who will speak; the model is not served as a chat model, which it is not. - The model the window serves is marked
in useand cannot be deleted. - A download goes on while the conversation does.
The model manager as a tool has Chat, Serve... and Inspect in place of Use this model: it opens other windows and leaves this one alone.

Connecting
Connect... links the window to something else, in place, without closing it. Its list is what was found, and only this machine is searched:
- Servers on this machine: whatever answers
GET /v1/modelson a loopback port from 7430 to 7480 (oklm serve, anotheroklm-gui), with its model's name, or "needs a key". - Saved servers: those connected to before by an address typed, and those kept with Save, each under its name when it has one.
- Agents: the agents started with
oklm agent --listen, found by the files they write for the user alone (agent-loop.md), with their model, project and session. A file whose process is gone is removed. - Sessions of this folder: the agent sessions kept
under
.oklm/sessionsof the folder the window was started in.
| Control | What |
|---|---|
| The list | a click selects a row; a server's address goes in the field, and a saved server's key, name, model and context in theirs |
| Server address | http://host:port or https://host:port, for
a server elsewhere |
| API key | the key the server asks for (for oklm serve, one of the
keys of its --api-keys file). It is shown masked, a
* a character, as it is typed and when a saved server is
selected |
| Context | for a server that is not OKLM's, which does not tell its context: the tokens the window counts on, 256 to 1 048 576; empty for 32768. An OKLM server tells its own |
| Name | what a saved server is called in the lists; its address names it otherwise |
| Model | for a server that is not OKLM's: the model the requests name, as the server lists it; empty for the first it lists |
| List models | asks the server at the address, with the key and the certificate
settings typed so far, for the models it lists
(GET /v1/models, oklm_client_remote_models()),
nothing else asked and nothing connected. They are offered in the
chooser beside the button, and the one chosen goes in the
Model field, where a name can still be typed. What
happened is one line above the fields, never with the key in it:
URL lists 3 models: choose one beside List models ...,
URL lists no model: type the name ...,
No server answered at URL: its models are not listed. The name can be typed.,
or
URL does not list its models: the server refuses the key (status 401: ...). The name can be typed. |
| CA file | a PEM file of the authority that signed the server's certificate, when it is not a public one |
| Do not check the certificate | for a self-signed certificate on a machine you trust: the connection is encrypted, the server not verified |
| Test | asks the address for its model and says what answered, without connecting |
| Connect | links the window to the selected agent or session, or to the server at the address once it answers |
| Use my own model | back to the model the window serves itself, which stays loaded all along |
| Refresh | looks again for servers, agents and sessions |
| Save | keeps the server in the fields (its address, its key, its certificate settings, its name, its model, its context) without connecting to it, so that one that does not answer now is kept too; a saved server of the same address is replaced |
| Forget | removes the selected saved server: its address, its key and what was kept with it leave the settings file |
A server reached by an address that was typed, or saved with
Save, is kept for the next time in
$XDG_CONFIG_HOME/oklm/gui-servers.json
(~/.config/oklm, %APPDATA%\oklm on Windows),
under the member servers: its url, its
key, its ca and verify, and, when
it has them, its name, its model and its
ctx. It is then listed under Saved servers
here and in the window a start
with nothing named opens on, where choosing it connects with the key
kept. A server found on this machine is not kept: it is found again.
The key is in that file and nowhere else. The file is created with
the owner's rights alone (mode 0600) before anything is written in it,
and is given them again each time it is written. On Windows there is no
such mode: the file, in the user's own %APPDATA% folder, is
given an access list of its owner alone when it is made and each time it
is written (oklm_file_owner_only(): a protected list, so
that nothing is inherited from the folder, with one entry that grants
the file's owner everything). The window never writes a key on its error
stream, in a note, a status line, a title or a screenshot: the field
masks it, a saved server's description reads
Key: ******** (kept with it), and the test aid that fills
the field (--press fill:key=...) writes
pressed: fill:key.
What the window is linked to decides what it does:
- A server: the conversation on screen is kept and goes on with the server's model. The Context slider asks that server.
- An agent: the window shows the agent's events from
its start (the messages, each tool call and the start of its result, its
answers) and the message box sends to it, also while it works. When a
tool call asks for permission the status line says what, and
Allow, Always and
Deny appear beside it; without an answer within two
minutes the agent refuses the call. Stop interrupts its
work. The agent keeps running when the window leaves it.
--agent PORTattaches at start (the token is read from the agent's file), which is how the sessions browser opens a live session. - A session: its messages, read only; nothing can be sent until the window is linked to something again.
A server that is not OKLM's
oklm-gui --url URL --key KEY,
Connect... and a saved server also reach a server that
only speaks the OpenAI API: llama.cpp's server, or any other with
GET /v1/models and POST /v1/chat/completions.
The address is http[s]://host[:port], with or without
/v1; any other path is the API's base and stands for
/v1 in each request, as the base_url of the
OpenAI SDKs does (oklm_client_new_api()). The hosted APIs are reached
that way, each with its key:
| Provider | Address | Notes |
|---|---|---|
| OpenAI | https://api.openai.com/v1 |
|
| Gemini | https://generativelanguage.googleapis.com |
any address of that host (/v1beta/models...) is read as
its OpenAI routes, /v1beta/openai |
| Anthropic | https://api.anthropic.com/v1 |
its OpenAI-compatible chat/completions; the key goes as
x-api-key too, for the list of models |
| Mistral | https://api.mistral.ai/v1 |
|
| DeepSeek | https://api.deepseek.com/v1 |
|
| Ollama | http://localhost:11434/v1 |
no key |
| any other | its base, the part before /chat/completions |
An endpoint given whole (.../v1/chat/completions,
.../v1/messages, .../v1/models) is read as its
base.
When it is linked to a server the window asks each of OKLM's own
routes once (GET /api/stats, GET /api/context,
GET /api/commands, GET /api/model, and
POST /api/compact with an empty count): a status other than
200, 404 and 405 among them, or an answer that is not OKLM's JSON, is a
server without that route, and it is not asked again as long as the
window stays linked to that server. A server with neither figures, nor a
context to tell, nor commands is not an OKLM server. The side panel then
shows, where the figures are:
Not an OKLM server: it gives no figures, no memory line, no model to load
in place of its own and no index, and does not tell its context.
The window counts on 32768 tokens, its own setting: the Context slider
changes it. A compaction counts by estimate.
Model asked of it:
models/qwen3-0.6b/Qwen3-0.6B-Q8_0.gguf
No error is shown and no request is repeated: the figures are not asked every two seconds, and nothing is said to be out of reach.
| What | On such a server |
|---|---|
| A conversation | as on OKLM: streamed, the reasoning a server gives apart
(reasoning_content, or reasoning) dimmed above
the answer, the tokens counted by the server when its stream ends with
them (usage, asked for the standard way with
"stream_options": {"include_usage": true}, as of
oklm serve). A server that refuses the member by name (a
400 or a 422) is asked again without it and no longer sent it. When a
reply ends without a count, the window counts the conversation by
estimate, three tokens for ten bytes of its system prompt and its
messages (the rate a compaction's estimate starts from): a note says so
once
(This server's replies end without a count of their tokens: the window counts the conversation by estimate, ...),
/stats and the line under the Context slider show that
figure, and the conversation is compacted by itself from it. Each
request names the model ("model") |
The figures, /stats |
the line above in place of the server's figures; /stats
has the conversation's own and that line |
Context, Apply context,
/context N |
the window's own setting, 32768 tokens unless another is kept with the saved server: applied at once, the server neither asked nor changed |
| The model | the first the server lists (GET /v1/models, read by
oklm_client_remote_models()), or the one kept with the
saved server (Model in Connect...). A server that lists
several has a chooser of them in the side panel, under the context's
line, in place of a name typed: the one chosen is asked for from the
next message and kept with a saved server. /model NAME does
the same for a name the server lists and refuses another
(lists no model named ...); /model alone and
/models list them, and the list above the message completes
/model with them. The list is read when the window is
linked, and again by /model, by /models and
when a name typed is not in it, so that a model the server lists later
is found. The button under Speech... carries the name.
Models... and its Use this model are
about this machine's models: the window starts again on its own
server |
| The model settings | the sampling, which goes with each request; everything else is
greyed, and the memory line reads
Not an OKLM server: it has no memory line and does not tell where it computes. |
/compact, a full context |
by estimate: the server counts no text
for the window. The request of the summary turns reasoning off with
chat_template_kwargs; a server that refuses the member (a
400 or a 422) is asked again without it, and no longer sent it |
Continue, /continue |
the reply is left open in the request, as on OKLM (llama.cpp's server goes on from it); a server that refuses that (a 400 or a 422) is asked once more with the reply closed and a message that asks to go on where it stopped, and the pieces after it go that way |
| A picture | sent as it is to oklm serve, the OpenAI way
(image_url with a data: URL). The server is
not asked first whether it takes pictures: one that does not refuses the
message, the reason is in the note under it, and the pictures are left
out of the next messages |
| A recording | sent as an OpenAI input_audio part:
{"type":"input_audio","input_audio":{"data":"<base64>","format":"wav"}},
before the text part. A WAV or an MP3 file goes as it is, with its
format; an Ogg, FLAC or Opus file, which that part does not name, goes
as the WAV of its samples (PCM, 16 bits, mono, 16 kHz).
oklm serve is still sent the file as it is. The server is
not asked first whether it hears: one that does not refuses the message,
the reason is in the note under it
(error 400: audio input is not supported by this model. The images, sounds and videos are left out of the next messages.),
and the sound is left out of the next messages, as a refused picture is;
Transcribe on it, Record,
/listen and Talk work with a Whisper model
of this machine, the window transcribing itself and sending the
text |
| A video | not sent:
cannot be sent: a server that is not OKLM's samples no video (its frames go as pictures, its sound as a recording),
its box framed in red |
| A document | read by the window and sent as text, as everywhere; one longer than the context is searched with this machine's embedding model |
Read aloud, /speak |
by the window itself, with a speech model of this machine |
| The sampling | temperature and max_tokens always;
top_k, top_p, min_p,
repeat_penalty and seed once they are set (the
model settings, /top-k...). A server that refuses one of
them by name (a 400 or a 422) is asked again without it at once, a note
says so once
(This server does not take top_k: /top-k has no effect on it, and is no longer sent to it.),
and no later request carries it; stream_options and
chat_template_kwargs go by the same rule, which is the
terminal chat's and the agent's backend's
(oklm_request_refused_member(), agent-loop.md).
A refusal that names no member the window sent is the message's error,
shown as the server worded it |
/rag, the server's index |
/rag cannot be done: no retrieval index here |
/decide |
A server that is not OKLM's has no /v1/decide route: it cannot be asked to choose.,
nothing asked |
The same window linked to oklm serve asks the same five
questions, gets five answers and behaves as before. A server that does
not answer at all when the window is linked (oklm-gui --url
on a server that is not up yet), or refuses the key, is taken for
neither kind: the first question is the only one asked, the side panel
says
The server cannot be reached: what it is, and what it can do, is found out when it answers.
(or The server refused the key.), and the context's control
stays greyed. The window goes on asking its figures every two seconds,
as it does of an OKLM server that is down, and nothing else. The first
request that gets through, the figures' or a message's, has the five
questions asked again before the next message goes: the status line says
The server answers now: it is not an OKLM server, and the window goes without what it lacks.
(or The server answers now.), and the side panel shows what
a link to that server shows.
Measured on llama.cpp's server (the test reference's build,
Qwen3-0.6B Q8_0 on the CPU, -c 4096,
--api-key), with tools/check_gui_remote.sh: a
message answered
(answered: 24 tokens in 0.60 s, the prompt 20 tokens), the
reply cut at 24 tokens gone on with by Continue in one
request that left it open (the prompt 44 tokens),
/context 4096 applied without a request, a conversation
compacted by estimate and the message after the note answered, the
server saved with its key and chosen in a window started with nothing
named. On the check's stand-in, which writes its requests down, each
window asks five requests under /api/ and none after them.
A picture, by hand on the same build: with SmolVLM-256M and its
projector, red-ball.png and
What is in the picture? are answered
(the prompt 149 tokens,
The image is a circle, ...); with Qwen3-0.6B, which reads
text only, the message is refused,
error 500: image input is not supported - hint: if this is unexpected, you may need to provide the mmproj. The images, sounds and videos are left out of the next messages.
A recording spoken to the window
(--mic-file samples/jfk.wav, /listen send) is
transcribed by the window itself with Whisper base in 0.5 s and answered
by the server.
The context size
The Context slider offers powers of two from 256
tokens to 1 048 576, with the context the model was trained with among
them, and Apply context gives the server that size
(oklm_server_resize() for the window's own model,
POST /api/context for a server at a URL). The model stays
loaded and the conversation stays on screen; the model reads it again on
the next message. Nothing changes while a reply is being written.
The server frees its cache and makes the new one, which it fills
whole, so a resize takes as long as that memory takes to touch. Measured
on SmolLM2-135M (23 KiB a token, four sequences): 0.14 s for a context
of 8192 tokens (0.7 GiB), 1.0 s for 65 536 (6 GiB), 1.8 s for 131 072
(12 GiB), and 61 to 64 s for 24 GiB and for 92 GiB on a machine that had
to give back what it kept of files to find them, the server's resident
memory growing by the whole cache. The resize therefore runs on a thread
of its own: the window goes on being drawn and used, the button reads
Resizing..., the line under it says
Resizing the context to 262144 tokens: 0:01 so far. The window goes on; nothing is sent until the server is done.
in amber, and the result is told when it ends, in the status line or in
the note of the /context that asked. Meanwhile a message is
not sent
(The context is being resized: the message is sent once the server is done.),
the link cannot be changed, and closing the window waits for the
server.
The line under the button follows the slider. It is the server's plan
for the size shown (oklm_server_context_plan(), or
POST /api/context with "dry_run": true for a
server at a URL):
- on the CPU the server's cache is in memory and grows with the
context, so the line gives the cache's size and the memory that is free:
for Qwen2.5-0.5B, 192 MiB at 4096 tokens, 768 MiB at 16 384 and 3.0 GiB
at 65 536. When it is more than what is free the line is red and the
button becomes Apply anyway. When it fits but takes
more than half of what is free the line is amber:
More than half of it: the server takes a minute or more to fill a cache this large.(/contextwrites that in a note before the wait, and a second note when it ended); - on a GPU the cache is what fits on the device, whatever the context, so the line gives what a conversation filling the size takes of it, and is red when the conversation would stop short, with the token count where the cache ends.
It is also red for a size larger than the context the model was
trained with (amber when the size is long to make as well: that is said
first). A server that does not take the request (an older one, or
--fixed-context) leaves the slider disabled.
On a server that is not
OKLM's the size is the window's own setting, since the server tells
none: 32768 tokens (what the agent takes for such a server), or what is
kept with the saved server (ctx of its entry, the
Context field of Connect...). The slider, Apply
context and /context N change it at once and ask
nothing of the server; the size is written back in a saved server's
entry, and lasts as long as the window for an address that is not saved.
It is what the window compacts by and what documents are fitted to. The
line under the button says so,
The window's own setting: this server does not tell its context, and is not asked to change it. With 8192 tokens the conversation is compacted near 6922.,
and is red when the conversation already holds more than the size shown:
The conversation holds 404 tokens: with 256 it will be cut at 256 tokens.
A server that refuses a prompt as longer than its context and tells that
context in the refusal (llama.cpp's n_ctx) gives the window
the true size:
The server refused the message and told its context, 4096 tokens: the window counts on that from now on.
A full context
The window deals with a full context by itself, as
oklm chat and the agent do; the rule, what a compaction
keeps and how a reply is gone on with are in inference.md. In the
window:
- Compaction. Past the share of the context where it
starts (230 tokens of a context of 512 with the window's reply length
and a small model), the next message is answered after the older
messages were summarized into a note,
Compacted, written where they end. The messages replaced stay in the conversation, and in what/exportand/savewrite; the model reads the note in their place, after the system prompt./compact [FOCUS]does it when asked. The summary is a request of its own to the server, with reasoning off; Stop ends it and leaves the conversation as it was. The window's own model counts the messages with its tokenizer. A server at an address is asked for the counts of its own tokenizer (countofPOST /api/compact, server.md): the messages of the plan in one request, then the texts of the summary's requests as they come. A server that does not answer it (one from before the member, another program's) leaves an estimate, the bytes at the rate the server last counted the conversation. - Continuing. A reply the context or the reply length
cut is gone on with in the same message: the request carries the reply
so far as its last message, left open (server.md). The window's
Reply length of 1024 tokens is then the size of one
piece. Moved, or set by
/max-tokens N, by the model settings or by what was kept for the model, it is the person's: a reply it cuts ends there, and a piece that goes on after a cut by the context has what is left of it. - Room. When the context is what cut the reply, or a
message does not fit in it, the window makes room: the next size the
Context slider offers, when the line under the slider
would have nothing to warn of for it (it fits the memory, the cache
holds it, the model was trained with it, it is not long to make), by the
resize above; else a compaction. A note says which
(
Context:The context was full (256 tokens): it is made larger, 512 tokens, and the reply goes on.). When nothing makes room the note says that the reply alone fills the context. - Continue. A reply that stays cut has a
Continue beside its name, after Speak:
it does as
/continue./continuealso answers a message that was not answered, once there is room. - Manual.
/autocompact offand/autocontinue offleave either to the person, as the two boxes of the model settings do (Compact when the context is nearly full, Continue a cut reply) and--auto-compact off,--auto-continue offat start./autocontinue Nsets the pieces in a row at most (8;--continue-max N). The three are kept in the window's settings file ("full_context") for the next window;oklm chatandoklm agentshare theirs in the user's settings (inference.md), which the window does not read. With compaction manual a full context says so in a note,Context fullfor a reply that was cut andNot answeredfor a message that does not fit, each naming/compact,/continueand the larger context; with continuing manual a cut reply saysThe reply was cut by the context: /continue goes on with it. - A reply stopped with Stop is not continued, and has no Continue.
- On a server that is
not OKLM's the context is the window's own setting, so the
window compacts by it and never asks for a larger one: room is a
compaction, counted by estimate. The server's own refusal of a prompt
longer than its context (
exceed_context_size_error,context_length_exceeded, or a sentence that names the context) is read as a full context; when it tells the server's context the window counts on that size from then on. A piece that goes on with a reply leaves the reply open, and is asked for in a message of its own (Go on with your last reply exactly where it stopped: ...) once the server refused that, for as long as the window stays linked to it. - Read aloud, and a spoken conversation. A reply that
is gone on with by itself is one reply for the voice: it is read
sentence by sentence as it is written, across its pieces, each word once
(the sentence a cut fell in is said once it has its end), and the times
a saved window writes are counted from the message. A note written
meanwhile (
Context,Compacted) is not read: the model does not read it either. When room is made by a compaction the note is inserted before the reply being read, which goes on being read. A reply that stayed cut was read to where it stops, the words of its last sentence included; its Continue, or/continue, has the piece that follows read from there, once the voice has said what it had (the sound made then is the piece's, so the reply's Speak makes the whole again). In a spoken conversation the window listens again once the whole reply was said. - Pictures. A message that carries a picture, a sound or a video stays where it is with its reply, whatever its age: a compaction summarizes the text messages around it, and the request after it still carries the file.

The picture is Qwen3-0.6B Q8_0 on the CPU in a context of 256 tokens, at temperature 0: the reply was cut after 227 tokens, the context was made 512, and the request that went on with it had a prompt of 256 tokens, the first prompt's 29 and the reply's 227, and added 81.
A request the server refuses for another reason, or a server that
cannot be reached, leaves a note under the empty reply,
Not answered, with the reason; the model is never sent the
note. It is written on the error stream too
(oklm-gui: not answered: ...).
A model that writes its reasoning in a channel (harmony, Gemma 4, Muse Glimmer) is continued by its server from the reasoning and the answer the window sends back, in the channel the reply was cut in (server.md).
Not done: the window was not run on a model that writes its reasoning
in a channel, the server's requests it sends were; a reply read aloud
while it is gone on with was not listened to; the window attached to an
agent leaves all of this to the agent, whose /compact,
/continue, /autocompact and
/autocontinue it passes on.
make gui-test runs these on the tiny model at
temperature 0 (tests): a reply the context cuts at
256 tokens is gone on with in a context made 512 and ends there; with
continuing manual its Continue is pressed and does the
same; a reply that does not end is continued once with
--continue-max 1 and says how to go on; a reply of the 8
tokens /max-tokens 8 asked for ends there; with compaction
manual a reply the context cut and a message that does not fit say so,
/compact makes two of three messages a note and
/continue answers the message; automatic, the same message
is answered after the compaction, with no Not answered; and
/compact after four exchanges makes six of eight messages a
note, the next message answered from a shorter prompt.
The model settings
The model's name, at the foot of the side panel, opens Model
settings (--model-settings: open at start): what
the model runs with, in the controls the server manager's form has for a
server (The Advanced window).
| Control | Option | What |
|---|---|---|
| Backend | --backend |
auto, cpu, opencl,
vulkan |
| Layers | --gpu-layers |
the slider's first place is as many as fit, then none to all the
model's layers. Greyed on the CPU, CPU beside it |
| Cache | -ctk, -ctv |
the cache quantized, keys and values at once: off
(f16), 8-bit (q8_0), 4-bit
(q4_0); other (Advanced) when the Advanced
window chose another pair |
| Cache size | --kv-mem |
the size of the cache the requests share, in MiB; empty is
auto: what is left beside the model on a device, four
contexts on the CPU (The memory
line) |
| MoE CPU | --n-cpu-moe |
a count of layers, or all. Greyed with the reason in
the field: not a MoE model,
the backend is the CPU |
| Context | -c |
the sizes of the side panel's slider |
| The line under it | the memory line of the model with
these settings, as the server manager's form has it, half a second after
the last change: it fits, it fits with a smaller cache, or, in red, it
does not fit. A model already served holds memory that a restart frees
before the settings are tried: the window asks its server what it holds
on its device and in the host's memory (device_memory.held
and host_memory.held of /api/stats, server.md), the plan counts
both as free, and the line says what was counted:
Counted as available: the 300 MiB the model served now holds in memory. On the device: 18.2 GiB of 18.3 GiB, 36 of 36 layers there. Counted as free there: 1.6 GiB free now and the 18.2 GiB the model served now holds, which a restart frees first.
What is held on a device is counted when the settings are for the same
backend, or for a device of the same size reached through another one
(one card seen by OpenCL and by Vulkan) |
|
| Temperature, Top-k, Top-p, Min-p, Repeat penalty, Seed, Reply length | --temp, --top-k, --top-p,
--min-p, --repeat-penalty,
--seed, -n |
the sampling. An empty field leaves the value to the server. It is
the window's own and goes with each request (temperature,
top_k, top_p, min_p,
repeat_penalty, seed,
max_tokens), so it applies at once, to a server at an
address too; the commands /temp, /top-k,
/top-p, /min-p, /seed and
/max-tokens write the same values |
| Advanced... | the Advanced window, here with
the tabs Cache, Sampling,
Vision, Speech,
Retrieval, Speed, MoE
disk and Other (the options without a
control). Its button Open in the server manager...
starts oklm-gui --tool servers on the model with the
settings shown and the Other field's text (--options),
where the Network tab is |
|
| Defaults | back to the settings the window has with nothing kept for the model: its command line's and the program's. The sampling at once; where the model computes after the question Apply asks, and what was kept for the model is forgotten with it. With a server at an address: the server's own sampling defaults again | |
| Apply | gives the window's own model what is not sampling. Grey when nothing
changed, and when the memory line says the settings do not fit (a
smaller cache fits: Apply works); pressed before the
line has caught up with a change, it waits for it
(What these settings take is being worked out) |
|
| Close | closes the window; what it shows stays, applied or not |
What Apply does. The window asks first:
Restart the model with these settings? The conversation is kept; the model loads again,
with the time the model took to load last when it is known and the
options that change, and three answers: Restart,
Cancel (the model runs as it did, the panel keeps what
it shows), and the box Do not ask again in this window,
after which Apply restarts without the question until
the window is closed. Then:
- a context alone is resized in place, as Apply context does (The context size): the model stays loaded;
- anything else restarts the model: the window's server is stopped, which frees the model and what it held on a device, the model is loaded again and served with the new settings, on a thread of its own, the window drawn meanwhile and nothing sent. The conversation is the window's, so it stays; the model reads it again on the next message. A reply being written is waited for, half a minute at most, then stopped, as the model switch does;
- settings that do not hold once loaded (a cache that the device has
no room for, a file that is not read) bring the settings before back:
These settings do not hold: ... The model is served as before; - the models beside the model are found before anything is stopped: a transcription or a speech model that is named and not installed refuses the restart with the command that downloads it.
The models that are options of the window. The
transcription and the speech models (--transcribe-model,
--speak-model, --speak-vocoder,
--speak-voice) are on the Speech tab, the
embedding and the reranking models of long documents
(--embed-model, --rerank-model) on
Retrieval, the projector (--mmproj) and
--ffmpeg on Vision, each field holding
what the option was given at start. The first group and the projector go
with Apply and its restart. The embedding and reranking
models are the window's own: they change at Apply
without a restart, and are refused while the conversation holds a long
document indexed with the ones before (New chat first).
Index reranker (--rag-rerank), on
Retrieval, is the window's server's: with an index
(--rag), Apply restarts the model with a
reranker that reorders the passages found in it, a file or an installed
model's name, and the side panel then says what the server tells of it
(rag.reranker of /api/stats).
What is kept for a model. The settings a model of
the window's own runs with are kept for it from a window to the next:
where it computes (Backend, Context,
the offload row, the Cache and Speed
tabs, Chat template, Picture side) and
its sampling. They are written, a second after they last changed and
when the window closes, in the window's settings file,
gui-servers.json in $XDG_CONFIG_HOME/oklm
(else ~/.config/oklm, %APPDATA%\oklm on
Windows), beside the servers connected to, under the model file's whole
path (a catalogue name and the file it stands for are one model):
"models": {"/models/qwen3-8b/Qwen3-8B-Q4_K_M.gguf": {"--gpu-layers": "20", "-ctk": "q8_0", "--temp": "0.3"}},
each option that is not the window's own start under its name; a model
that runs with nothing else has no entry. They are read when the window
serves that model again: at start, before the model is loaded, and when
a model is loaded in place of another (/model, Use
this model): its sampling is the window's at once, and where it
computes is what the model is loaded with, in one load and without the
question Apply asks (The
Models window); settings the memory line refuses leave the model
loaded with the window's alone.
A model loaded in place of another does not run with what was chosen for the model before. The options are of two kinds:
| Kind | Options | At the switch |
|---|---|---|
| the model's | Layers (--gpu-layers), MoE
CPU (--n-cpu-moe), the expert tier
(--expert-gpu), Cache size
(--kv-mem), the expert cache, what is pinned and the pack
(--expert-cache, --pin-gb,
--expert-pack), the draft model and its tokens
(--draft, --draft-n), Chat
template, Picture side
(--tiles) |
back to automatic: as many layers as fit, the cache that is left, the file's template, no draft model. They were chosen for another model's depth, experts, vocabulary or projector |
| the machine's, or the user's in this window | Backend, Context, the cache's
types (-ctk, -ctv: off, 8-bit, 4-bit),
--devices, the sequences, --spec-ngram,
--prefix-dir, the sampling (/max-tokens 8, a
slider, the panel) |
as the window has them |
A context longer than the one the new model was trained with is
reduced to it, and the note of the switch says so. What is kept for the
new model is then read over the result, option by option, the model's
options included. An option of the model's that the command line named
(oklm-gui -m A.gguf --gpu-layers 20) was given for the
model named with it: at a switch it goes back to automatic as the same
option set in the panel does, the note of the switch lists it with the
others (automatic again: --gpu-layers), and from then on it
is an option like the others, read from what is kept for a model and
kept when the panel sets it. Switching back to the first model does not
bring it back, unless it is kept for that model. The memory line judges
the settings before the model before is stopped, as it does for kept
settings: when they do not fit, what is kept is left aside and the
window's settings are tried alone, and when those do not fit either the
model is not loaded and the model before goes on serving, the reason
said. After the switch the panel shows what the model runs with (the
slider on as fit, an empty Cache size),
and what the model before ran with is kept for it. Which wins:
- an option of the machine's or the user's given on the command line
(
-c,--backend,-ctk,-ctv,--devices,--spec-ngram,--prefix-dir) is that window's, for every model it serves: what is kept for it is neither read nor replaced; - an option of the model's given on the command line
(
--gpu-layers,--n-cpu-moe,--expert-gpu,--kv-mem,--expert-cache,--pin-gb,--expert-pack,--draft,--draft-n,--chat-template,--tiles) is the window's for the model named with-m, and what is kept for that model is neither read nor replaced for it; at a model loaded in place it is automatic again, and what is kept for the other model is read; - what is kept wins over the program's defaults (0.7, 1024 tokens,
auto, 4096 tokens); - at a model loaded in place of another, what is kept for the new model wins over the window's current values, option by option; for the rest, the model's options are automatic again and the others are the window's current values, not its start's;
- settings kept that no longer hold when the window starts (a device that is gone, a draft model that moved) are said on the error stream and the model is served with the window's own;
- the projector, the models of speech and of retrieval and the index are the command line's, not kept;
- Defaults goes back to the window's own and forgets the model: the command line's and the program's, less the model's options the command line named once another model was loaded in place.
A server at an address. Where it computes is its
own: the backend, the layers on its devices, its context and its cache's
types are shown as the server tells them (GET /api/context,
server.md), greyed, and
the tabs say Set by the server the window talks to. Its
sampling defaults (oklm serve --temp, --top-k,
--top-p, --min-p,
--repeat-penalty, --seed, -n, or
what the server manager's Sampling tab set) are the window's when it
attaches: they fill the fields and the Temperature and
Reply length sliders, and go with the window's requests
until a field is changed, which is then the window's own for as long as
it is attached. A server's defaults win over what is kept: at each
attach the sampling starts from the window's own start, not from what a
model of the window's has kept, then takes what the server tells; a
server that does not tell them (an older one, another program's) leaves
the window's own start. Nothing is kept for a server's model: its
settings are its command line's, which the server manager's profiles
keep. Back on the window's own model (Use my own
model), its sampling is the one it had.
The Other field of the chat window
The Advanced window's Other tab is a text field for
the options of oklm serve that have no control, as on a
command line. Apply reads it with the table the
controls are made from:
- an option that has a control goes to its control
(
--temp 0.3 -ctk f32typed there set Temperature and Cache keys), where the memory line judges it with the others; - of the rest, the window's own server takes those that say how it
works,
--workers N,--prefill-chunk N,--prefill-share F,--deterministic,--swap-mem MB,--spec-guard RATE,--prefix-min N,--prefix-disk-mb MB,--ingest-keep N,--speak-max N,--preload,--observe,--observe-lens K, and the expert cache's,--expert-usage FILE,--expert-mirror FILE,--expert-loaders N,--expert-predict off|last|router,--expert-direct,--no-expert-direct(with Expert cache set on MoE disk), with the valuesoklm servetakes; - anything else is refused, nothing changed, the status line naming it
(
Other options: --insecure is not an option the window's own server takes there (how it works and the expert cache's are; a control has the others, or the server manager).,Other options: --workers takes a number from 0 to 1024, not many.,Other options: a quote is not closed.): the names, the keys, the adapters, the formats and the network of a server are not the window's to set, and-c,--backendand the form's fields have their own place in the panel.
A change of the field is a change like the others: the line under the
sampling reads
Changed, not applied: -ctk, the other options, the question
asked before a restart names it
(What changes: -ctk, the other options), and the model
restarts with them
(The model was restarted in 0.1 s with these settings (context 512, -ctk f32 --workers 3 --deterministic)).
They are kept for the model with the rest, as the member
other of what the settings file keeps for it, read by the
next window on that model and at a switch to it (a model loaded in place
with nothing kept runs with none), and Defaults forgets
them. The field is greyed for a server at an address.
--settings 'OPTIONS' leaves what has no control in the
field, as typed.
Limits. The window's own server takes the options a chat window's
model can use: --fixed-context and --rag-dir
are greyed there
(for a server other programs reach: in the server manager),
and the Other field takes the options listed above, not every option of
oklm serve. The embedding and reranking models are refused
while the conversation holds a long document, and compute on the CPU:
their backends (--embed-backend,
--rerank-backend) are a server's, greyed here
(the window's own: on the CPU). The memory line counts what
the server loads beside the model, and a speech model that is only
installed as the window will open it (The
memory line).
A retrieval index
--rag INDEX goes with a model the window serves itself:
its server opens the index (as oklm serve --rag INDEX does,
rag.md) and every message is sent
with "rag": true, so it is searched in the index and
answered from the passages found, with the server's defaults (hybrid
search when the index has keyword postings, 5 passages, corrective
retrieval). The line under the message box says so at start. The
retrieval manager's Open a chat window starts the
window this way (the example
there).
/rag changes it from the message box.
/rag off stops searching the messages in the index (the
side panel reads Index: docs.gguf (off)),
/rag docs.gguf, the open index's name, uses it again; both
at once, the server untouched. /rag other.gguf opens
another index, or a first one in a window started without
--rag: the window starts again on it, and the conversation
is not kept. The index is opened once before that, so that a file that
is no index is refused with the reason and the window stays as it
is.

Options
| Option | What |
|---|---|
-m MODEL |
a GGUF file, or the name of an installed catalogue model: the window serves it itself |
-c N |
the context of the window's own server, tokens (4096) |
--backend cpu|opencl|vulkan|auto |
where the window's own model runs |
--gpu-layers N, --n-cpu-moe N|all,
--expert-gpu GB|auto, --kv-mem MB,
-ctk TYPE, -ctv TYPE,
--devices N, --draft MODEL,
--draft-n N, --spec-ngram N,
--prefix-dir DIR, --expert-cache GB|auto,
--pin-gb GB, --expert-pack FILE,
--chat-template FILE|NAME, --tiles SIDE |
with -m, or for the model chosen in a window started
with nothing named: where the window's own model computes, as
oklm serve takes these options and as the model settings show them at start.
Those of the model's (the layers, the experts, the cache's size, the
expert cache of a model read from disk, the draft model, the template,
the picture's side) are for the model named with -m:
automatic again when another model is loaded in place (which wins) |
--parallel N |
sequences of the window's own server (2) |
--url URL |
a server elsewhere, http://host:port or
https://host:port |
--key KEY |
that server's API key |
--ca FILE |
a private authority, for a server whose certificate it signed |
--no-verify |
accept any certificate (a self-signed local one) |
--agent PORT |
attach at start to the agent listening on that port |
--rag INDEX.gguf |
with -m: a retrieval index opened beside the model |
--mmproj FILE |
with -m: the projector the window's server opens, a
GGUF file or the name of an installed catalogue model; without it, the
catalogue's for the model named, when it has one |
--auto-compact on|off |
the older messages are summarized into a note by themselves when the
context is nearly full (on), or only with /compact; over
what the window's settings file keeps (a full
context) |
--auto-continue on|off |
a reply the context or the reply length cut is gone on with by
itself (on), or only with /continue and its
Continue |
--continue-max N |
the pieces a reply is continued with in a row at most (8) |
--attach FILE |
a file that goes with the first message; repeatable, eight at most (Files with a message) |
--ffmpeg |
an attached video in a container other than Ogg Theora is decoded
through an ffmpeg subprocess, by the window (its thumbnail
and length) and by the window's own server, as
oklm serve --ffmpeg does; OKLM_FFMPEG=1 allows
it too |
--rerank-model NAME|FILE, --no-rerank |
the reranker the passages found in a long document are put in order
with: an installed catalogue model's name or a GGUF file; without it,
the first installed reranker of the catalogue; --no-rerank:
none, the index's order |
--no-doc-cache |
a long document's index is a temporary file of this window, built anew each time, not kept in the cache folder |
--embed-model NAME|FILE|hash |
the embedding model a document longer than the context is indexed
with: an installed catalogue model's name, a GGUF file, or
hash; without it, the first installed embedding model |
--chat-file FILE |
a saved conversation opened at start: what /save,
/export json, oklm chat --chat-file and the
dashboard write |
--model-settings |
the model settings open at start |
--settings 'OPTIONS' |
a test aid: options of oklm serve set in the model
settings once the --send given before it on the command
line were answered, the sampling at once, the others applied as
Apply and Restart do, those without a
control being the Advanced window's Other field;
repeatable, four at most. --settings defaults presses
Defaults; --settings 'ask OPTIONS' leaves
the question open, for the window's picture |
--press PRESS |
a test aid, given once for each press and done in its turn after the
--send before it on the command line: a button pressed
without a person. speech opens the Speech window;
models the Models window, models:NAME on a
catalogue model, as a click on its row; connect the Connect
window, connect:PORT with the server or the listening agent
of this machine on that port selected; link presses its
Connect, own its Use my own
model; choose opens the window that asks what to talk
to, choose:PORT with the server or the agent of this
machine on that port selected, choose:URL with the saved
server of that address and choose:NAME with the installed
model of that name, as a click on its row; choose:file
selects the row that chooses a model's file and
choose:mmproj the row of its projector,
choose:file:PATH and choose:mmproj:PATH being
the file dialog answered with that path (the file listed and its row
selected); use presses its button, which connects to the
server, attaches to the agent, loads the model in the window or opens
the file dialog; advanced opens the Advanced window of the
model settings and advanced:TAB opens it on a tab
(cache, sampling, vision,
speech, retrieval, speed,
moe, other); continue presses the
Continue of the reply that was cut;
context:N drags the side panel's Context
slider to the size nearest N, not applied, and
apply-context presses Apply context;
fill:FIELD=VALUE writes a field of the Connect window as a
person types it (url, key, name,
model, ctx, ca; the window writes
pressed: fill:FIELD, never the value),
list-models presses its List models and
pick-model:NAME chooses a model among those listed,
save presses its Save and
forget:URL selects the saved server of that address and
presses Forget; model:NAME chooses a model
in the side panel's chooser of the models a server that is not OKLM's
lists; paste is Ctrl+V; export and
export:json are /export and
/export json as a person types them, which open the
system's dialog for the file (a scripted --send "/export"
writes chat-DATE.md without asking), the press after one
waiting for its dialog to be closed; attached:N presses
nothing and is done once N files are attached to the message being
written, dropped or pasted meanwhile, so that the --send
after it goes with them (two minutes at most). The window writes
oklm-gui: pressed: ... on the error stream |
--shot-when STATE |
a test aid, with --screenshot: the window is saved the
first time it is in the state named, not once it is at rest:
resizing (a context being resized), recording
(a recording being made, a second and a half in), speaking
(a reply being read aloud), talking (a spoken conversation
whose reply is being said), commands:TEXT (TEXT put in the
message once the scripted messages are answered, the list of commands
open above it). It says
the window is STATE: saved in that state, or, when the
state never came,
the window was never STATE: saved as it is |
--open-servers FILE.png |
a test aid: once the settings of --settings are in the
panel, the Advanced window's Open in the server
manager... is pressed, the server manager being started with
--save-profile opened --screenshot FILE.png: it keeps its
form as the profile opened, saves its window and ends |
--transcribe-model NAME|DIR |
with -m: the Whisper model the window's server
transcribes with, a checkpoint directory or the name of an installed
catalogue model; without it, the smallest installed one. Attached to an
agent, the one the window loads itself (Speech
to text) |
--no-transcribe |
no speech to text: the window's server loads no Whisper model |
--transcribe-language CODE|auto |
the language spoken (en, fr, ...), given
with each transcription, or auto for the recognizer to find
it; without it, the language chosen last in the Speech window, else
found |
--transcribe-translate |
what is said is written in English whatever the language spoken
(Whisper's translation task; a server's
/v1/audio/translations) |
--mic NAME |
the capture device recorded, by the name PulseAudio or PipeWire
gives it (pactl list short sources); without it, the
system's default input |
--transcribe-attached |
transcribe an attached recording the model cannot hear without a press on Transcribe; the sound track of an attached video too, on a link that takes its frames |
--listen-silence SECONDS |
end a recording after that long without a word (off); the pause that ends a turn of a spoken conversation (1.2). The Speech window's slider and box |
--listen-threshold DB |
the level above which the microphone hears speech, dB of full scale, -60 to -5 (-40) |
--listen-send |
send the message once a recording's text is in it (off) |
--talk |
a spoken conversation from the start (A spoken conversation) |
--no-interrupt |
speaking does not stop a reply's voice: for loudspeakers |
--no-partial |
no words shown while a recording is still coming: the recognizer is left alone until the recording ends |
--mic-file FILE[@TIME] |
a test aid: record this sound file in place of the microphone.
Repeated, with @SECONDS or @voice+SECONDS: the
turns of a scripted spoken conversation |
--speak-model NAME|FILE |
with -m: the speech model the window's server reads
aloud with (loaded at the first reading), a GGUF file or the name of an
installed catalogue model; without it, an installed one is loaded by the
window at the first reading. With --url or
--agent: the one the window loads itself (Replies read aloud) |
--speak-vocoder FILE |
with --speak-model FILE: the vocoder's GGUF, which a
catalogue name finds by itself |
--speak-voice FILE|NAME |
a speaker file of OuteTTS in place of the built-in voice: the one
the Voice: chooser starts on. With --url,
when that server speaks: the name of one of its voices |
--speak-speed X |
how fast replies are read, 0.25 to 4 (1): the Speech window's Speed of the voice at start |
--speak-whole |
a reading is played once its whole sound is made: the Speech window's box, on at start |
--no-speak |
nothing is read aloud: no speech model is loaded, by the server or by the window |
--read-aloud |
Read aloud on at start: every new reply is read while it is written |
--speak-file FILE.wav |
a test aid: write what would be played to this WAV file and play nothing |
--models |
open the Models window at start |
--send TEXT |
send one message at start, once the files of --attach
are read; a text that starts with / is a command and runs
as one. Repeated (eight at most), each goes a second after the one
before was answered, or ran. With --screenshot, the window
is saved once the last one's reply and the next figures are there, or a
second after a command ran or the window said why the message did not
go |
Threads. Requests run in worker threads: one for the reply (the client's stream), one for the figures, one for each attached file while it is read, one for the documents being indexed and searched, one for a sound being transcribed, one for a text being made into sound, one feeding the stream that plays it and one for a resize of the context. The drawing thread only reads what they wrote, under a lock, and redraws when something changed (a 30 Hz timer). Each message keeps the height it measured at the conversation's width, measured again only when its text or the width changes, and only the messages in view are drawn. Closing the window cancels a reply in progress and stops the embedded server.
Limits. The window does not list the sources of an answer from an
index; --rag is the window's own server's, another server
or an agent is not asked to search. An image in a reply is not loaded.
One conversation at a time, not kept when the window closes unless
/save or /export wrote it
(--chat-file, or a drop, opens it again). A command typed
while the window is attached to an agent and that the window does not
have is sent as a message: what the agent makes of it is the agent's. On
Windows the window is the same; an agent listens there too
(oklm agent --listen, since 2026-11-13).
Run on 2026-11-13 with oklm-gui.exe under Wine 11 and
Xvfb, not on Windows itself: the four unattended runs of
make gui-test (a message answered by the tiny model, a PDF
attached by --attach, /help, a recording of
--mic-file transcribed); SmolLM2-135M served by the window,
its reply streamed; /he showing the list of commands, Tab
completing, Enter writing the help's note; a text attached by its path
typed and Enter, and by Attach, whose dialog is Wine's
and gave the file back, then read and answered from
(The document is about the lighthouse keeper, Maren Okoye.);
the Models window listing the catalogue; Connect... on
a server started apart with oklm.exe serve (its address
typed, Test, Connect, a message
answered), and --url; --agent PORT on
oklm.exe agent --listen (the panel
Attached to an agent, a message answered,
The agent's turn ended (success)); a server with a
certificate nobody signed refused
(certificate verify failed), accepted with
--ca and with --no-verify; two files dropped
on the window (an X drop made by tools/xdnd_drop.py, which
Wine hands on as the system's), one named
note été Ελληνικά 日本語.txt, both in the row of files and
sent with a message; /export and /export json
through the system's dialog, conversation.md and
conversation.json written under the names it proposed, and
its question before a file is replaced; a picture sent to SmolVLM-256M's
projector (a red ball: Red.) and a sound to Ultravox's
(jfk.wav:
The speaker says: "And so, my fellow Americans ask not what your country can do for you, ask what you can do for your country.",
30 tokens in 30 s on the CPU). Defects seen and fixed: the window's own
model was named by its whole path when the path was written with
backslashes; a file dropped on the window did nothing (the build before
refused the drop: Allegro has no drop on Windows, see Files with a message); the dialog of
the exports opened without the name it was given (Allegro's dialog
passes the system an empty name: on Windows the tools call the system's
dialog themselves, gui_save_dialog(), which proposes the
name, opens in its folder and asks before a file is replaced). Seen and
left: the window's font has no glyph for Chinese, Japanese or Korean, a
name in those is drawn with gaps.
The model manager
The catalogue of models.md in a window:
what OKLM offers, by category, what is installed, and for the selected
model its description, size, context and licence. It does what
oklm models list, pull and rm do,
with the download's progress and the licence to read before agreeing,
and opens a chat, a server or the inspector on a model.
oklm-gui --tool models # the manager in a window of its own
oklm-gui --tool models --select qwen3-8b # that model selected at start
oklm-gui --tool models --installed # the installed models only
Menu entry: OKLM Models.

Example: download a model, then delete it
With an empty model directory (OKLM_MODELS_DIR names it;
the last line of the window says which is used):
- Start
oklm-gui --tool models --select smollm2-135m. The description ends withNot installed.; Download is the only button enabled, and the last line reads0 of 57 models installed, 0 MiB. - Press Download. The row's state becomes
downloading, the bar counts (87 MiB / 138 MiB 100.1 MiB/s) and the button reads Cancel download. Pressed then, it stops the download; what arrived is kept and the next Download goes on from it. - A second later here (138 MiB), the row reads
installed, the descriptionInstalled.andsmollm2-135m downloaded., the last line1 of 57 models installed, 138 MiB. Delete, Chat, Serve... and Inspect are now enabled. - Press Delete: the button reads Really delete? and nothing is removed yet. Selecting another row lets go of the question.
- Press it again: the file is removed, the last line says
smollm2-135m deleted.and the state is empty again.


Example: a model whose licence asks for agreement
- Start
oklm-gui --tool models --select llama3.2-1b. The description names the licence (llama3.2) and the address of its text, and ends withThis licence has to be agreed to before the download. - Download is greyed until I have read this
licence and agree to it is ticked. The answer is recorded as
oklm models pullrecords it, once a model.

| Control | What |
|---|---|
| The list | the catalogue by category (general chat, coding, images, sounds,
security, retrieval): name, size on disk, the context the model was
trained with, and the state: installed,
downloading or nothing |
| Installed only | keeps the installed models (and the one being downloaded) |
| Refresh | reads the model directory again: a model installed or deleted
elsewhere (oklm models pull, another window) then
shows |
| The description | the selected model: what it is, its size and files, its context (the
default, the range -c takes, the limit), its licence and
where the licence's text is, whether it is installed; then how the last
download ended |
| I have read this licence and agree to it | shown for a model whose licence is more than a permissive one, until it was agreed to |
| The bar | the download: bytes on disk, the total, the rate |
| Download | installs the selected model, its files checked against their SHA-256; pressed again (Cancel download), stops: what arrived is kept and resumed. One download at a time |
| Delete | removes the selected model's files, after a second press (Really delete?) |
| Chat | a chat window serving the model:
oklm-gui --tool chat -m NAME |
| Serve... | the server manager with the model in its form:
oklm-gui --tool servers -m NAME |
| Inspect | the model inspector on the model's file:
oklm-gui --tool inspect -m NAME |
| The last lines | how many models are installed, what they take and where they are kept; then what the last press did |
Chat and Serve... are offered for
the text models (chat, coding, security). Chat is
offered for the vision and audio models too (SmolVLM, Ultravox), which
the chat window serves with their projector. Whisper and the retrieval
models are used from the command line and by the retrieval manager,
which the description says. For a speech-synthesis model (OuteTTS)
Chat reads Read aloud: it opens a chat
window with Read aloud on
(oklm-gui --tool chat --read-aloud), which asks what to talk to and whose
replies are then read with it; Serve... is off, the
description naming oklm serve --speak-model NAME.
| Option | What |
|---|---|
--select NAME |
a model of the catalogue selected at start |
--installed |
the installed models only (the box ticked) |
Limits. The catalogue is the program's own; a model that is not in it
is a file given to a tool with -m. The models are kept
where oklm models keeps them (models.md). The same manager opens inside the
chat window (the Models window), where
Use this model changes the model that window serves. A
build without TLS cannot download, and says so. On Windows the window is
the same. Run on 2026-11-13 with oklm-gui.exe under Wine 11
and Xvfb, not on Windows itself, with a model directory of its own
(OKLM_MODELS_DIR): the catalogue listed;
smollm2-135m downloaded from Hugging Face (144811360 bytes,
the server's certificate checked against the roots of the certificate
store, which Wine fills from the machine's), marked
installed; Chat, Serve...
and Inspect each starting its tool on it
(oklm-gui.exe --tool chat -m smollm2-135m and the two
others); Delete and its confirmation, the folder
emptied. The store of a Windows machine was not read.
The server manager
On the left a form that starts oklm serve in the
background; on the right every server of the machine, and what can be
done with the selected one. It is for keeping servers running without a
terminal each: they are started through the registry of background
processes (server.md), so
they go on when the window is closed, and another window, or
oklm serve --list, finds them again. A server can also be
kept across logins: it is then a service of yours that
the system starts at each login and starts again when it fails (server.md), and the
list shows it whether it runs or not.
oklm-gui --tool servers
oklm-gui --tool servers -m qwen3-8b --backend cpu -c 8192 # the form filled at start
oklm-gui --tool servers --profile NAME # a kept profile loaded in the form
oklm-gui --tool servers -m qwen3-8b --options "--gpu-layers 20 -ctk q8_0 -ctv q8_0 --temp 0.6 --queue 32" # options of oklm serve read into the form's controls
oklm-gui --tool servers --profile NAME --advanced sampling # the Advanced window open on a tab
Menu entry: OKLM Servers.

Example: start a server from the form
- Start
oklm-gui --tool servers -m smollm2-135m --backend cpu -c 4096: the model, the backend and the context are in the form. (From an empty form: choose the model in Installed,cpuin Backend.) - Click Sequences and type
2. Half a second later the line under the Context slider reads4096 tokens on cpu, a cache of four contexts shared by the requests: 612 MiB of memory (weights 136 MiB, cache 360 MiB), then the memory available andIt fits., in grey: the server's cache in host memory is four contexts' worth whatever the number of sequences that share it (Cache size, left empty, isauto). - Type
small-cpuin the field beside Profile and press Save profile: the form is kept under that name, and the list beside it offers it from now on. - Leave Port empty and press Start
server. The line under the button gives the port, the pid and
the command:
Started on port 7430 ... oklm serve -m smollm2-135m --backend cpu -c 4096 --parallel 2 --port 7430. - Once it answers the server is listed and selected:
7430 SmolLM2-135M-Instruct- 0/2 ... 4096and its pid. Under the buttons are its address, since when it runs, its figures and its command line; under Log, the log's file and its first line,serving SmolLM2-135M-Instruct-Q8_0 on http://127.0.0.1:7430/. - Send it a request from anywhere (Chat opens a
window on it). The log gets the request's line
(
POST /v1/chat/completions 200 in=36 out=8 50ms) and the figures the tokens a second and the requests answered: the picture above, where the window was started again on the profile and the server selected.
Example: layers on the device, a quantized cache, sampling defaults
- Start
oklm-gui --tool servers -m qwen3-8b: Backend isauto. - Under Backend, drag Layers from
as fitto20/36and choose8-bit (q8_0)in Cache. The line under Context follows:4096 tokens on vulkan, a cache of 44.9 contexts shared by the requests, its keys Q8_0 and its values Q8_0: 8.9 GiB of memory (weights 2.2 GiB, cache 6.0 GiB), 118.4 GiB available. On the device: 10.5 GiB of 18.3 GiB, 20 of 36 layers there, as asked. It fits with 20 layers on the device, the others on the CPU.The cache is the one the server would make: on a device, what is left beside the model, here 44.9 contexts of 4096 tokens, the rows of the 16 layers on the CPU in host memory. Typing2048in Cache size makes it 2.0 GiB (a cache of 2.0 GiB as asked, 7.5 contexts). MoE CPU is greyed,not a MoE modelwritten in it. - Press Advanced.... On Sampling,
type
0.6in Temperature and20in Top-k; on Network,32in Requests waiting. MoE disk reads dimmed: pressed, its fields are greyed and the line under them saysGreyed: --expert-cache, --pin-gb, --expert-pack: not a MoE model.Press Close. - The line beside Advanced... now reads
-ctk q8_0 -ctv q8_0 --temp 0.6 --top-k 20 --queue 32: what the window has set. - Press Start server:
Started on port 7430 ... oklm serve -m qwen3-8b --gpu-layers 20 -ctk q8_0 -ctv q8_0 -c 4096 --port 7430 --temp 0.6 --top-k 20 --queue 32. - Save profile keeps all of it; a chat window opened on the server (Chat) shows 0.6 and 20 in its model settings.


Example: act on a listed server
The list also holds what answers on the loopback ports without being
in the registry: here a chat window started with
oklm-gui -m smollm2-135m --backend cpu -c 2048, which
serves its model on port 7431.
- Select the server on port 7430 and drag the Context
slider on the right to 2048, then press Apply context.
The row's context becomes 2048, the line at the bottom left says
The server on port 7430 now has a context of 2048 tokens. Its conversations are kept and read again on their next message.and the server's notice is in the log. - Click the row of port 7431. The text under the buttons says what it
is:
Found by its port: your own window (oklm-gui, pid ...) serving its model. Stop closes that window, after asking once more; it cannot be started again from here.Restart is greyed, and there is no log: its output is where it was started. - Press Stop. The button reads Really
stop? and the line at the bottom left
Stop your oklm-gui on port 7431 (pid ...), closing its window? Press "Really stop?" to end it.Nothing is stopped yet. - Press Really stop?: the chat window closes and its row goes.
- Select the server on port 7430 and press Stop. It
is in the registry and answers no request at that moment, so it stops at
once;
oklm serve --listthen prints(no server is running).

Example: a server that comes back at login
A server started as above ends with your session. To find it again after the next login, keep it.
- Fill the form as in the first example
(
-m smollm2-135m --backend cpu -c 4096), and type a port in Port,7440: a kept server should be where its clients expect it, login after login. - Check Keep across logins. A field appears beside
the box with a name made from the model's,
smollm2-135m; typesmol-cpuin its place. The name is the service's: lower case letters, digits,-and_. - Press Start server. The line under the button reads
Kept as smol-cpu and started on port 7440, loading smollm2-135m: it starts at each login, and again when it fails. Its unit: /home/me/.config/systemd/user/oklm-serve-smol-cpu.serviceand the command. Once it answers the server is listed and selected like any other, withsmol-cpuin the last column, KEPT AS; the text under the buttons haspid ..., kept across logins as smol-cpu since ...and the unit's file, and Log follows~/.local/state/oklm/services/smol-cpu.log. - Press Stop. The button reads Really
stop? and the line at the bottom left
Stop the kept server smol-cpu? It stays kept and starts again at your next login, unless you remove it from the kept ones. Press "Really stop?" to stop it.Press it: the row stays,stoppedin place of its figures, the text under the buttons saysKept across logins as smol-cpu (port 7440, smollm2-135m): stopped. It starts at your next login, or with Start., and Restart now reads Start. - Log out and in again, or press Start: the server
runs again, its log goes on in the same file after the line
server stopped. Restart stops it and starts it again in one press. - To change what is kept, change the form (move
Context to 8192, say), select the server in the list
and press Re-keep with form, beside Start
server. The button reads Really change? and
the line
Change the command kept as smol-cpu to this form's settings? Its unit is written again under the same name, and the server is started again. Press "Really change?" to do it.Press it:The command kept as smol-cpu is changed, and the server started again on port 7440, loading smollm2-135m, with the unit's file and the new command. It is whatoklm serve --rekeep smol-cpu ...does: the name, the log and the start at login stay, and a server that was stopped is not started (It was not running and is not started: it has the new command at its next start). The name beside the box is not read here: the server keeps the one it has. An empty Port keeps its port. - Give the form a name beside Profile and press Save profile: the profile remembers the box and the name, so the same server can be kept again later from the list of profiles.
- When the server is no longer wanted at login, select it and press
Remove from kept. The button reads Really
remove? and the line says what goes:
Remove smol-cpu from the kept servers? It is stopped, its unit (...) and its log are deleted, and it no longer starts at login.Press it: the row goes, andoklm serve --keptprints(no server is kept).
A kept server that cannot run is listed all the same, with the
reason: kept with a model file that does not exist, its row reads
starting while the service manager tries again
(its command ended with the status 1; it is started again in a moment),
then failed
(it failed five times within a minute and is no longer started again),
and its log, below, has the server's own words
(cannot load ...). Start tries once more;
Remove from kept removes it.
To have kept servers started when the machine boots, before anyone logs in, see lingering in server.md: it is a setting of the system's that the window never changes.
The form
Each field is an option of oklm serve (server.md); an empty one leaves the option
out.
| Control | Option | What |
|---|---|---|
| Profile, the name beside it | the profiles kept; choosing one puts it in the form. The name is the one Save profile keeps the form under and Delete profile removes | |
| Save profile, Delete profile | keep the form under the name typed, in place of the profile of that name; remove the profile named | |
| Installed | the installed models of the catalogue that answer chat requests (models.md); choosing one writes its name in Model | |
| Model | -m |
an installed model's name, or a GGUF file's path |
| Backend | --backend |
auto (left out), cpu, opencl,
vulkan |
| Port | --port |
empty: the first free loopback port from 7430 to 7480; a port that is taken is refused before anything starts |
| Layers | --gpu-layers |
the offload row, under Backend: the layers on the
device, the model's last ones with its output head. The slider's first
place is as fit (the option left out: as many as fit in the
device's memory), then none, a count over the model's
layers, all. Greyed on the CPU, CPU beside
it |
| Cache | -ctk, -ctv |
the cache quantized, keys and values at once: off (f16)
leaves both options out, 8-bit (q8_0),
4-bit (q4_0). other (Advanced) shows when the
Advanced window chose the two types one by one; choosing one of the
three here sets both again. No switch goes with it: OKLM has no
flash-attention option to turn on |
| MoE CPU | --n-cpu-moe |
for a mixture-of-experts model on a device: the routed experts of
the first N layers, or of all, stay in host memory and run
on the CPU. Greyed otherwise, the reason in the field
(not a MoE model, the backend is the CPU) |
| Cache size | --kv-mem |
the offload row's last control: the size of the cache the requests
share, in MiB. Empty is auto (the option left out): what is
left beside the model on a device, four contexts on the CPU. A size is
the cache's, whatever the room; one that holds less than one context is
refused by the memory line. -1, read from a command line,
is the server's contiguous cache (the whole context for each sequence).
It is the only place of the option: the Advanced window's
Cache tab no longer has it |
| Sequences | --parallel |
requests decoded at once, beside Cache size; empty:
the option is left out and oklm serve takes as many as its
cache holds, 512 at most (server.md). The
cache is shared by them: its size does not change with their number |
| Context | -c |
powers of two from 256 tokens to 1 048 576, with the context the model was trained with among them |
| The line under the slider | the memory line: what the server takes with that context, that backend, those layers on the device, that cache and those experts on the CPU, against what the machine has free, computed half a second after the last change. Grey when it fits, with the cache whole or smaller; red when it does not, when the model is not found, when a value is not one its option takes, or when the context is larger than the one the model was trained with | |
| Keys file | --api-keys |
a file of API keys, one a line, readable by its owner only |
| TLS cert, TLS key | --tls-cert, --tls-key |
both or neither |
| Adapters | --lora-module, --lora |
NAME=FILE words are served beside the model; one file
alone is applied to it |
| Index | --rag |
a retrieval index (rag.md) |
| Advanced..., the line beside it | the Advanced window: the other
options a tab a group, and a text field for every option without a
control. The line is what the window has set, as it goes on the command
line
(-ctk q8_0 -ctv q4_0 --temp 0.6 --model-id 'my model'),
nothing set otherwise; pointed at, it is shown whole |
|
| Keep across logins, the name beside it | oklm serve --keep NAME |
checked: Start server makes the server a service of
yours under that name (a systemd user unit,
oklm-serve-NAME.service), started now, at each login and
again when it fails, its output in
~/.local/state/oklm/services/NAME.log. The name field is
there while the box is checked, filled from the model's name when it is
empty: 1 to 48 lower case letters, digits, - and
_. A name already kept, a key given inline in
More (--api-key ...) and a system without
systemd's user manager are refused, with the reason under the button,
and nothing is started; where servers cannot be kept, checking the box
says so at once. A profile remembers the box and the name |
| Start server | runs the command in the background, its output in a log of the
registry's directory. The line under the button gives the port, the pid
and the whole command: the form's options, the offload row's and the
Advanced window's in the order of oklm serve's table, then
the adapters, then the other options as typed. An option that does not
apply is left out (--gpu-layers on the CPU,
--n-cpu-moe for a dense model); a value that is not one its
option takes is named instead
(--top-k takes a whole number from 0 to 10000000.) and
nothing starts. The server is listed as starting while it
loads its model, then selected, and its log is shown from its first
line; when it ends before answering (a file that is not a model, a port
taken meanwhile) the line says so and the log stays, with the
reason |
|
| Re-keep with form | oklm serve --rekeep NAME |
for the kept server selected in the list: asks once more (Really change?), then gives it the form's command under the name it has. Its unit is written again in place, it stays started at login and keeps its log; it is started again when it ran, and a stopped one has the new command at its next start. Port left empty is the port it has; another port has to be free. The box and the name beside it are not read. Refused as Start server refuses (a key given inline, a field that does not hold), the unit then as it was. Grey unless a kept server is selected |
The profiles are in
$XDG_CONFIG_HOME/oklm/gui-server-profiles.json (else
~/.config/oklm, %APPDATA%\oklm on Windows), a
file only the user can read, since the options may hold what is not for
others. A profile keeps the form's fields under the names it always had,
what the offload row and the Advanced window set as an object
options (each option under its own name,
{"--gpu-layers":"20","-ctk":"q8_0","--temp":"0.6"}), and
the other options as the text more. A profile kept before
the options had controls is read as it is: the options its
more text holds go to their controls
(--queue 8 --temp 0.4 to Requests waiting
and Temperature), and what the table does not know
stays in the text, untouched.
The Advanced window
One window, in the server manager (Advanced... in
the form) and in the chat window (the
model settings): the options of oklm serve that are not
in the form, a tab a group. It is 740 by 420 pixels, and so fits in a
tool's window 760 pixels wide, the server manager's smallest: nine tabs
of 80 pixels in the server manager, eight in the chat window, a label's
column of 250 pixels and the control after it. A field left empty, a
chooser on its first line and a box left clear leave the option out, and
the field shows in grey what the server does without it
(0.8, as many as fit,
the model's own).
| Tab | Controls and options |
|---|---|
| Cache | Cache keys -ctk and Cache
values -ctv, one by one: f16 (the
keys' default; the values follow the keys), f32,
bf16, q8_0, q4_0,
q4_1, q5_0, q5_1, and
iq4_nl (slower on a device) last; The context
cannot be resized by a client --fixed-context;
Devices --devices; Expert tier
(GiB) --expert-gpu (a size, or
auto) |
| Sampling | the defaults of what a request leaves out:
Temperature --temp, Top-k
--top-k, Top-p --top-p,
Min-p --min-p, Repeat
penalty --repeat-penalty, Seed
--seed, Reply length -n. The
seed is that of every request that names none, so that the same request
gives the same text, and the length that of a request that gives none;
without them a server draws a new seed for each request and lets it run
until its sequence is full (server.md). The chat
window sends its own with each request |
| Vision | Chat template --chat-template;
Projector --mmproj; Picture
side --tiles; Videos through
ffmpeg --ffmpeg |
| Speech | Transcription model --transcribe-model
and Its backend --transcribe-backend;
Speech model --speak-model, Its
vocoder --speak-vocoder, Its
voice --speak-voice, Its backend
--speak-backend |
| Retrieval | Embedding model --embed-model and
Its backend --embed-backend;
Reranking model --rerank-model and
Its backend --rerank-backend;
Passages kept --rag-k;
Search --rag-mode; Index
reranker --rag-rerank; Folder of
indexes --rag-dir |
| Speed | Draft model --draft and Tokens
a draft --draft-n; Prompt lookup
--spec-ngram; Prefix cache folder
--prefix-dir |
| MoE disk | a mixture-of-experts model read from disk: Expert cache
(GiB) --expert-cache (a size, or
auto), Pinned (GiB) --pin-gb,
Expert pack --expert-pack. In both tools:
the chat window's own server reads its experts from disk with them, and
its command line takes the three |
| Network | Address --host; The model
cannot be changed by a client --fixed-model;
Requests waiting --queue; Largest
upload (MiB) --upload-max. The server
manager's |
| Other | a text field for every other option, as on a command line, quotes
keeping a value with blanks whole
(--model-id 'my model' --workers 12; on Windows the quotes
of its command lines, --model-id "my model"): added to the
command after the controls' own, so a --port there replaces
the field's. In the chat window the field takes the options its own
server can be given (the
Other field of the chat window) |
What does not apply is greyed, with the reason as
text. An option that needs something the form does not have is
greyed and says why: a text field in its own place
(no projector for this model, no speech model,
no draft model, no embedding model,
no reranking model, no retrieval index,
no expert cache, the backend is the CPU,
not a MoE model), and the line at the foot of the tab for
all of them
(Greyed: --tiles, --ffmpeg: no projector for this model.).
A tab none of whose options applies has its name dimmed and the reason
as its tooltip; it can still be opened and read. What decides: the
backend chosen, the other fields (a projector named, a speech model
named), the catalogue (a projector it has for the model), and the memory
plan's reading of the model's file (its layers, whether it routes
experts); until that reading is there nothing is held against an option.
A greyed option is not written on the command line, whatever its field
holds.
One table. Each control is a line of one table
(apps/gui/serve_opts.h): the option's name and its other
spelling (-ngl, --cache-type-k,
-ncmoe, -md, -np,
--model), what its value is (a whole number, a number, a
text, one of a few words, a switch), its range, what the server does
without it, its tab, and when it applies. The command line, a profile's
options, the line beside Advanced..., the
memory plan's parameters and the reading of --options or of
an old profile's text all go through it. 53 options have a control; the
others of oklm serve stay in the Other
field: --workers, --prefill-chunk,
--prefill-share, --deterministic,
--observe, --observe-lens,
--model-id, --insecure,
--api-key, --swap-mem,
--lora-scale, --spec-grammar,
--spec-guard, --prefix-min,
--prefix-disk-mb, --embed-model-id,
--rerank-model-id, --transcribe-model-id,
--speak-voices, --speak-model-id,
--speak-max, --preload,
--ingest-keep, --gpu-stream,
--template-vars, --tool-format, the
--rag-* switches other than the four above, and the
--expert-* options other than the three of MoE
disk.
The servers
The list is asked again every two seconds, off the drawing thread:
the servers of the registry (those this window started, those of another
window, those started from a terminal), and whatever answers as a server
on the loopback ports 7430 to 7480, which the chat window's Connect
looks at too (a chat window's own server, a server outside the
registry). Only this machine is searched. Each row has the port, the
model, the requests being decoded over the sequences, those waiting, the
tokens a second, the context, and the pid of a registered server or of
your own program found behind the port (GET /health,
/api/stats, /api/context). The servers kept
across logins are asked of the service manager at each pass too
(oklm_kept_list()): one that runs is the registry's row,
with its name in the last column, KEPT AS; one that
does not run has a row of its own, with stopped,
failed or starting in place of its figures,
the port and the model of its command line, and its log.
| Control | What |
|---|---|
| Refresh | asks the servers at once, looks at the ports and reads the installed models again |
| The list | a click selects a server |
| Stop | SIGTERM, then SIGKILL after ten seconds. A registered server
(oklm_procreg_stop()): asks once more (Really
stop?) while requests are being answered. A server found by its
port that is a program of yours, an oklm or an
oklm-gui of your user holding the listening socket
(oklm_procreg_port_owner(), read from /proc:
Linux): always asks once more, naming the program and its pid, and a
chat window serving its model closes with it
(oklm_procreg_stop_listener(), which looks for the process
again and leaves alone a pid that no longer holds the port). Anything
else behind a port (another program, another user) is not stopped from
here, and the text under the buttons says so |
| Restart | a registered server only: stopped, then its command line started again in the background, in the directory it ran in and on the same port; a server started from a terminal has a log from then on |
| Stop, for a kept server | through the service manager (systemctl --user stop:
SIGTERM, then SIGKILL after fifteen seconds), always after asking once
more, since stopped is not removed: the server stays kept and starts
again at the next login |
| Restart, Start, for a kept server | through the service manager too. The button reads Start for a kept server that does not run (stopped, or failed: it is then tried again, its count of failures given back) |
| Remove from kept | a kept server only: asks once more (Really
remove?), then stops it, disables and deletes its unit, deletes
its log; it no longer starts at login. Only a unit oklm
wrote is touched |
| Dashboard | its page in the system's browser (server.md) |
| Chat | a chat window on it, a process of its own:
oklm-gui --tool chat --url ADDRESS, with --key
when the server asks for one and --no-verify for a server
of this machine on TLS |
| The text under the buttons | the selected server: its address, since when it runs, its figures, its command line; or what it is when it was found by its port; for a kept server, the name it is kept under and its unit's file, and, when it does not run, its state with the service manager's reason and whether it starts at the next login |
| Context, Apply context | another context without starting it again
(POST /api/context, server.md); the line under
the slider is the server's own answer for the size shown
(dry_run), red when it does not fit, when a conversation of
that size would stop where the cache ends
(It stops near 97664 tokens, where the cache ends.) or when
it exceeds the trained context; the button then reads Apply
anyway for a size the server would refuse. The size the server
has is asked as another is, so that a server whose cache ends before its
own context says it here as the chat window
attached to it does: the two lines are the same answer of the same
server |
| API key | for a server that asks one: sent to the servers of this machine that refused their figures without it. A server started with a keys file is asked with that file's first key, read from its command line; the typed key is not kept |
| Log | what a server started in the background wrote, followed as it grows
(oklm_procreg_log_read()), the last 48 KiB kept; the text
can be selected and copied. A server started from a terminal has none
here |
The memory line
The line under Context, in the form and in the chat
window's model settings, is the memory plan of a server
(oklm plan --serve, memory-plan.md) given
-c, --backend, --gpu-layers,
-ctk, -ctv, --n-cpu-moe,
--devices, --parallel and
--kv-mem as the controls have them. It judges as the server
does, because the cache's size comes from the rule the server applies
itself (oklm_kv_cache_rule(), one function for both): the
weights go first, then the cache takes what is left beside them on a
device (four contexts on the CPU), or the size Cache
size gives. The verdicts:
| Verdict | When | The line |
|---|---|---|
| it fits | the cache holds four contexts or more, or the size asked for, and the host's and the device's memory hold what the server takes | 4096 tokens on vulkan, a cache of 23.8 contexts shared by the requests: ... It fits.
(a cache of four contexts on the CPU) |
| it fits with a smaller cache | on a device, what is left beside the model holds one context at least and less than four: the server starts and serves, and keeps fewer conversations of that length at once | 40960 tokens on vulkan, a cache of 2.4 contexts shared by the requests: ... On the device: 18.2 GiB of 18.3 GiB, 36 of 36 layers there. It fits with a smaller cache: 2.4 contexts where four are kept when there is room.
Grey: Apply and Start server go
on |
| it fits with fewer layers on the device, for the cache | the device holds the whole model with less than one context of cache beside it, and neither the layers nor the cache's size were set: the most layers that leave the device one context go there, the others and their cache stay in memory, when memory holds them. Slower than the whole model on the device, and a conversation that fills the context is held | 131072 tokens on vulkan, a cache of 1.0 contexts shared by the requests: ... On the device: 18.2 GiB of 18.3 GiB, 28 of 36 layers there. The device holds the model with less than one context of cache beside it: 28 of its 36 layers go there with 14.3 GiB of the cache, the other 8 keep 4.1 GiB of it in memory, slower than the whole model on the device. It fits with 28 layers on the device, the others on the CPU, and a smaller cache: 1.0 contexts where four are kept when there is room.
Grey |
| it fits with some layers on the device | the model is larger than the device (its weights do not go there
whole), or --gpu-layers keeps some on the CPU: the cache is
sized for the layers that go there, the others keep theirs in
memory |
4096 tokens on vulkan, a cache of 1.2 contexts shared by the requests: ... On the device: 3.7 GiB of 3.7 GiB, 22 of 36 layers there. The model is larger than the device: 22 of its 36 layers go there with 407 MiB of the cache, the other 14 keep 259 MiB of it in memory. It fits with 22 layers on the device, the others on the CPU, and a smaller cache: 1.2 contexts where four are kept when there is room.
Grey |
| it does not fit | the cache would hold less than one context with the layers or the size that were asked for (a conversation filling the context could not be kept), or the host's or the device's memory is short | in red:
It does not fit: what is left for the cache holds 0.7 contexts, less than one; a context of 131072 tokens takes 18.0 GiB.
(every layer asked for on the device), or, with a size,
that cache holds 0.5 contexts, less than one, or
the system would page the weights or end the server, or,
when the device is short of a context beside the whole model and memory
of what fewer layers there would leave it,
the device has not one context of cache beside the whole model, and memory has not the room for the layers and the cache that leaves it |
The line names the cache's types when they are not f16,
the layers on the device (as asked when the slider chose
them), the layers whose experts stay on the CPU, and a size asked for
with the contexts it holds
(a cache of 4.0 GiB as asked, 1.8 contexts). With
--kv-mem -1 (the contiguous cache) it reads
the whole context kept for each of N sequences. Checked on
2026-11-15 on an RX 7900 XT, on OpenCL and on Vulkan, with Qwen3-8B
Q4_K_M (make gui-device-test, below): at 40960
tokens, which the line refused before (27.3 GiB for four contexts
against 18.3 GiB), it says a smaller cache of 2.4 contexts, 97664
tokens, and the server started from the form tells the same 97664 in
cache_tokens (GET /api/context); at 131072
tokens it said It does not fit, and since 2026-11-16 it
says the 28 layers that go on the device with a context of cache, which
the server started from the form has ("n_gpu_layers":28,
cache_tokens 133664), It does not fit being
what it says with every layer asked for there (GPU
layers at 36). The line under a selected server's
Context slider is the server's own answer for the
layers it has: a server that runs moves none, so its line stays red
above its cache where the form, which starts another, is not.
What shares the memory. The line counts what the
server loads beside the model, each part where the server places it, and
takes it out of the cache's room before the cache is sized, as the
server does (oklm_kv_cache_rule() is given the same two
figures by both: server.md). It then reads
Counted beside the model: 590 MiB on the device., or
5.3 GiB on the device, 457 MiB of memory:
| Part | Option | On the model's device | In host memory |
|---|---|---|---|
| the expert cache of a mixture of experts read from disk | --expert-cache, --pin-gb (MoE
disk) |
the cache's slots and what is pinned, as the context sizes them
(oklm_context_plan()), in place of the routed experts'
mapping:
The routed experts are read from the file: an expert cache of 117 MiB is counted in place of their 697 MiB. |
|
| the draft model | --draft |
the context the server gives it
(oklm_serve_draft_params()): its layers, its buffers and a
cache of five contexts of the model's length |
its layers left on the CPU, its buffers there |
| the projector's encoder | --mmproj, else the catalogue's for a model given by its
name; --tiles |
when the model's backend is a device: its weights and the activations of the largest picture the server accepts, or a chunk of sound | on the CPU backend: the file's weights and the arena its activations
are computed in (oklm_vision_host_bytes_at(),
oklm_audio_host_bytes()) |
| the transcription model | --transcribe-model,
--transcribe-backend |
when its backend is the model's device: its weights and its passes' buffer | on the CPU (its backend's default): the checkpoint's weights, the
two arenas of its passes and the copies it makes of its weights
(oklm_whisper_host_bytes()) |
| the speech model and its vocoder | --speak-model, --speak-vocoder (else the
catalogue's), --speak-backend |
when its backend is the model's device: the language model's weights, cache and activations, the vocoder's network | on the CPU: the language model's weights with the cache, the
activations and the scratch of the context the synthesizer opens, the
vocoder's weights with its copies and the longest call of its network, a
whole prompt's codes (oklm_speech_host_bytes()) |
| the backend's own buffers | their reserve (64 MiB on OpenCL, 352 MiB on Vulkan) when an encoder or a sound model shares the device | ||
| the embedding and the reranking models | --embed-model, --embed-backend;
--rerank-model, --rerank-backend |
when its backend is the model's device: its context there, weights,
activations and cache (oklm_context_device_bytes()) |
on the CPU (their backends' default): its weights and its context,
cache, activations and scratch, as the embedder and the reranker make it
(oklm_embed_context_params(),
oklm_rerank_context_params()); on the device: what the
context keeps in host memory |
Each part is sized by the function the server sizes it with once
loaded (oklm_server_beside_plan()), from its file, with
nothing placed. What computes on the CPU is counted with what it works
with there, not its weights alone: bge-small-en-v1.5 Q8_0, a file of 35
MiB, takes 112 MiB with the context of 64 texts the embedder gives it;
Whisper base, 277 MiB, takes 609; OuteTTS 0.2 with its vocoder, 636 MiB
of files, 1.3 GiB; SmolVLM-256M's projector on the CPU, 181 MiB, 284.
The draft model's context was counted already. What cannot be sized is
named instead of being left out:
Not counted: the speech model (no vocoder named), the draft model (its file cannot be read).
The other reason is no such backend here.
A model beside the model on a device that is not the
model's (a transcription, a speech, an embedding or a reranking
model whose backend is another device; or any device, when the model is
on the CPU) is counted on that device, and that device is judged for it:
On vulkan1: 94 MiB of 84.1 GiB, for the embedding model.
When the device has not the room, the line is red and says which device
is short, the model's own memory holding or not:
It does not fit: vulkan1 has not the room for what is asked of it.
The choosers of these four backends (Its backend in the
Speech and Retrieval tabs) offer cpu, opencl,
vulkan and auto, and the other devices this
machine's backends have, by their names in oklm devices
(vulkan1); a device named on the command line
(--embed-backend vulkan1) gets its line in the chooser
too.
In the chat window the projector and the
transcription model are those found for the model when no option names
one: the window's server is given both. A speech model that is only
installed is not the server's: the window opens it itself, on the CPU,
the first time a reply is read aloud or a spoken conversation starts.
Until then the line counts it as the window will open it and says so
(Counted beside the model: 1.8 GiB of memory. Counted: the speech model, loaded at the first reading.),
so that turning the voice on later does not take memory the line gave to
the model; once the window has loaded it, it is in the machine's own
figures and no longer counted apart; with --no-speak there
is none. The embedding and the reranking models a long document is
indexed and ordered with are the window's own too, on the CPU: those an
option names, else those the window finds by itself (the first installed
of the catalogue's retrieval models, as it will choose them when a long
document is attached). The latter are counted as what the window will
load, and the line says so
(Counted: the models a long document is searched with, loaded when one is attached.),
until the window has loaded them; with --embed-model hash
and --no-rerank there is none.
Checked on 2026-11-16 on an RX 7900 XT
(make gui-device-test, below), a server
started from the form each time:
| Server | The line | The server |
|---|---|---|
| SmolVLM-256M with its projector, 8192 tokens, Vulkan | Counted beside the model: 590 MiB on the device; a
cache of 806992 tokens |
cache_tokens 806992; device_memory:
vision 249934848 and backend_own 369098752,
the 619033600 bytes the line counted |
| the same, OpenCL | 302 MiB on the device; 820096 tokens |
820096; vision 249934848 and backend_own
67108864, the line's 317043712 |
| Qwen3-8B Q4_K_M with Qwen3-0.6B Q8_0 as its draft model, 8192 tokens, Vulkan and OpenCL | Counted beside the model: 5.3 GiB on the device, 457 MiB of memory;
a cache of 59072 tokens, 7.2 contexts |
59072 on both |
Qwen3-0.6B Q8_0 with bge-small-en-v1.5 Q8_0 on the device
(--embed-backend), 8192 tokens, Vulkan and OpenCL |
Counted beside the model: 94 MiB on the device, 26 MiB of memory;
a cache of 160352 tokens |
160352 on both; device_memory.embed 98637824, the bytes
the line counted there; host_memory.embed 14433792, the
bytes the line counted for what that context keeps in host memory |
| Qwen3-0.6B Q8_0 with bge-small-en-v1.5 Q8_0 and MiniLM-L6 F16 on the CPU, 8192 tokens, Vulkan and OpenCL | Counted beside the model: 232 MiB of memory; a cache of
161216 tokens, the model's alone |
161216 on both; host_memory: embed
81582080 and rerank 80681984, the 162264064 bytes the line
counted for their two contexts |
Without these parts counted the line planned 833872 tokens for the first server (3.2 % more than the server made on Vulkan, 1.7 % on OpenCL) and 97664 for the third, 11.9 contexts where the server keeps 7.2. The embedding model on the device was counted in host memory, by its file's 35 MiB, and the cache planned as the model's alone; the two models on the CPU were counted as their files, 79 MiB, where they take 232 with their contexts.
Options
| Option | What |
|---|---|
-m MODEL |
the form's Model at start |
--backend cpu|opencl|vulkan|auto |
the form's Backend at start |
-c N |
the form's Context at start |
--profile NAME |
a kept profile loaded in the form |
--options 'OPTIONS' |
a line of options of oklm serve read into the form at
start: each option that has a control goes to it, the adapters to their
field, the rest to the Advanced window's Other field.
What the chat window's Open in the server manager...
starts the tool with |
--save-profile NAME |
the form as it is at start (after --options) kept under
that name, as Save profile does |
--advanced [TAB] |
the Advanced window open at start, on cache,
sampling, vision, speech,
retrieval, speed, moe,
network or other |
--press PRESS |
a test aid, with --screenshot, given once for each
press: they are done in turn once the form is shown, and the window is
saved after the last. start: Start server
is pressed and the server waited for. keep:NAME: the
Keep across logins box is checked and given its name.
profile:NAME: the form is saved under that name
(Save profile). The presses on a server name it by its
port or, for a kept server, by its name (T), and select it first:
select:T; stop:T, Stop
pressed once; restart:T, Restart (which
reads Start for a kept server that is stopped);
unkeep:NAME, Remove from kept pressed
once; rekeep:NAME, Re-keep with form
pressed, then pressed again once it asked; ask-rekeep:NAME,
pressed once; dashboard:T and chat:T.
context:N drags the selected server's
Context slider to the size offered nearest N, applies
nothing, and writes the line under it once the server answered for that
size
(oklm-gui: the context line: 131072 tokens, cache 97664 tokens, red: ...;
the chat window writes the same line for its own slider). A press that
leaves a question is followed by yes (the button, which
then reads Really ...?, pressed again) or
no (the row chosen again, which withdraws the question, as
a click on it does) |
With --screenshot the tool writes on the error stream,
before it ends, the command the form makes
(oklm-gui: the form's command: oklm serve ...), what the
Advanced window has set and the memory line: how the form is checked
without a person. With --press it adds what the window said
(oklm-gui: asked: ..., oklm-gui: said: ...)
and, for a server started, its command line as the registry has it
(oklm-gui: the server: port ..., pid ...: oklm serve ...),
its own account of its settings
(oklm-gui: its context: {...}, the answer of
GET /api/context) and whether that agrees with the form
(oklm-gui: the server agrees with the form: --backend -ctk -ctv -c --fixed-context --temp ...,
or differs from the form: --temp=0.8): the backend, the
context, the layers on the device, the cache's types, each sampling
default given, and whether the context can be resized.
Limits. Only this machine is searched; a server elsewhere is reached
from the chat window. Keeping a server across logins needs systemd's
user manager (Linux); elsewhere, and on Windows, the box says why not
and Start server starts nothing while it is checked.
tools/check_gui_kept.sh drives these buttons with
--press, a window a step, so that no place of the form is
in it: by default against a stand-in service manager
(OKLM_SYSTEMCTL, tools/standin_systemctl.py)
whose units and services are the script's own, nothing of the user's
being read or written (21 checks, passed on 2026-11-15);
REAL=1 runs the same steps against systemd's user manager,
for a person to start (21 checks, passed on 2026-11-16 with systemd 262,
the unit removed and the user's units as they were). Checked on
2026-11-13 under Xvfb with systemd 262's user manager and the test model
(the script as it then was, clicking the form; fifteen checks): the box
and its name, Start server leaving a unit of mode 600,
enabled and active, the server answering and in the registry once with
its service; Stop asking first, then the service
inactive and still enabled; Start and
Restart, another process each time on the same port;
Re-keep with form asking first, then the unit holding
the form's command under the same name and the server started again with
it (another process, the same port, the new --model-id
served), and a stopped server's command changed without starting it; a
profile keeping "kept":true and the name; Remove
from kept asking first, then no unit, no log and no process; a
kept server whose model is missing shown starting then
failed with the manager's reasons; and a window whose
systemctl does not answer, where the checked box starts
nothing. The window works from about 760x480; the form takes two fifths
of a narrow window and its rows are closer in a short one. On Windows
the window is the same: Start server,
Stop, Restart and the log work through
the registry there. Keep across logins is offered there
too since kept servers have a Windows half (a value of the user's
Run key: started at login, not again when it fails, server.md); the box
and the kept servers' buttons were not driven under Wine,
oklm.exe serve --keep and its companions were. Run on
2026-11-13 with oklm-gui.exe under Wine 11 and Xvfb, not on
Windows itself: a server of the test model started from the form on a
port typed, listed with its figures and its pid, its log followed, then
Stop and its confirmation, the log ending with
server stopped and the list empty. Then, the same day and
the same way: the context of a running server changed from 256 to 8192
with Apply context (the log's line, the row's figure);
Restart (another pid on the same port, a log of its
own); a profile saved, written in
%APPDATA%\oklm\gui-server-profiles.json, and loaded by
--profile; a server started with a keys file (401 without
the key, 200 with it, the row saying API key, the key typed
in the field); a server started apart and taken out of the registry,
listed as
Found by its port: your own oklm (pid 32), not in the registry,
ended by Stop and Really stop? (the owner
of the port is read in the system's table of listening sockets, which
Wine 11 fills), its log ending with server stopped. A
server of another program of this machine, on Linux, was listed by its
port without a pid, as it should. Dashboard handed the
server's address to the system's opener
(http://127.0.0.1:PORT/, read from a stand-in for
xdg-open, which Wine starts), and Chat
opened the chat window on it
(oklm-gui.exe --tool chat --url ..., its panel showing the
server's context).
The conversion tool
Makes a GGUF file from a Hugging Face checkpoint, or another GGUF
file from a GGUF file, with the commands of convert.md: oklm convert,
oklm quantize and, to judge the result,
oklm ppl. The window does not convert anything itself: it
shows the command lines, starts them in the background, and reads what
they write. What it adds to the commands is the size of the result
before anything is written, and the perplexity of the source and of the
result side by side.
oklm-gui --tool convert
oklm-gui --tool convert -m model.gguf --type Q4_K_M
oklm-gui --tool convert -m hf_dir --type Q4_K_M --text text.txt --run --check
Menu entry: OKLM Convert.

Example: requantize a GGUF file and measure the loss
- Start
oklm-gui --tool convert -m models/smollm2-135m/SmolLM2-135M-Instruct-Q8_0.gguf --type Q4_K_M --text text/shakespeare.txt(or type the source, choose the type in the list and type the text's path: 12 kB of prose). - Under Source the window says what the file is:
GGUF file, 138 MiB ... Architecture llama (supported), 30 blocks, context 8192 ... 134.5 million parameters in 272 tensors ... Types now: 61 F32, 211 Q8_0 ... Verdict: loadable. - Result is proposed beside the source,
SmolLM2-135M-Instruct-Q4_K_M.gguf, and the estimate readsThe result will be 101 MiB (105454432 bytes, 272 tensors): 73 % of the source's 138 MiB, with the free space of its folder: the picture above. - Command lines shows what the buttons run: one
oklm quantize, and twooklm ppl. - Press Convert (
--runat start presses it). The bar counts the bytes written; at the end it readsdone, and the file is 105 454 432 bytes, as estimated. - Press Measure perplexity (
--check).oklm pplruns on the source, then on the result, its windows in the log; then the two lines readSource: perplexity 33.821, 138 MiBandResult: perplexity 35.342, 101 MiB ...: +4.50 % against the source.
Chat with the result then opens the new file in a chat window, and Inspect it in the inspector.

Example: a Hugging Face folder to Q4_K_M
- Start
oklm-gui --tool convert -m models/smollm2-135m-hf --type Q4_K_M --text text/shakespeare.txt -c 256. The source readsHugging Face checkpoint, 257 MiB of weights in 1 safetensors file ... Architecture LlamaForCausalLM, written as llama ... Types now: 272 BF16 ... Verdict: convertible (272 GGUF tensors). oklm convertdoes not write Q4_K_M itself, so the command lines are two:oklm convert ... --out models/smollm2-135m-hf-Q4_K_M-F16.gguf --type f16, thenoklm quantizeof that file. The estimate saysTwo steps: an F16 file first, then Q4_K_M from it. Not estimated yet: the size is computed from the F16 file's tensors once it exists.- Press Convert: the two commands run one after the
other. The F16 file (270 885 664 bytes) is kept beside the result, 105
453 856 bytes; the estimate now reads
101 MiB (105453856 bytes, 272 tensors): 39 % of the source's 257 MiB. - Press Measure perplexity:
oklm pplscores the checkpoint itself, then the result, with 256-token windows:Source: perplexity 38.737,Result: perplexity 40.329 ...: +4.11 % against the source.

The controls
| Control | What |
|---|---|
| Source | a GGUF file (any shard of a split model) or the folder of a
checkpoint (config.json and safetensors weights), typed or
chosen with Browse... |
| the lines under it | what the source is: for a GGUF file what
oklm info --json says (oklm_model_info_json():
size, name, architecture and whether OKLM runs it, blocks, context,
parameters, tensors per type, the file type, the verdict and its notes);
for a checkpoint its architecture and the GGUF one it is written as,
blocks, context, parameters, tensors per type, whether it has a
tokenizer, and whether oklm convert takes it (a dry run to
F16) |
| Result | the file written; proposed beside the source, named after it and the
type (model-Q8_0.gguf gives
model-Q4_K_M.gguf), until another is typed |
| Type | the 36 file types oklm quantize writes, the usual ones
first; the line under it and the list's hint say the bits a weight and
what the type is for |
| One type everywhere (--pure) | oklm quantize --pure: no rule keeps the sensitive
matrices in a larger type |
| the name field | general.name of the result
(oklm convert --name,
oklm quantize --override-kv general.name=str:NAME); empty
keeps the source's |
| the importance matrix field, Browse... | oklm quantize --imatrix, a file of
oklm imatrix |
| the estimate | the size the result will have, to the byte, its share of the source's, the free space of its folder (a warning when it is short), and whether the file exists already |
| Command lines, Copy | the exact commands the buttons start, a line each, quoted for a shell: they run the same in a terminal |
| Convert, Stop, the bar | start the conversion; stop the running command (its unfinished
.tmp file is removed); the bytes written of the file's
size, or the windows scored |
| Chat with the result, Inspect it | the result in the chat tool or in the inspector, a window of its own
(oklm-gui --tool chat -m FILE,
--tool inspect -m FILE) |
| Quality: text, Browse..., window, Measure perplexity | oklm ppl -f TEXT -c WINDOW --backend cpu on the source,
then on the result; the text must hold at least one window (512 tokens
by default) |
| the two lines under it | the perplexity and the size of the source and of the result, and the change in percent |
| the last line and the box under it | what is going on, and the running command's log, followed as it is written |
The file browser (the three
Browse... buttons) lists a folder, folders first: for
the source the folders and the GGUF files, with Use this
folder to take the folder shown as the checkpoint (the window
says when config.json is there); for the text and the
importance matrix every file. A double click, or Open,
shows a folder or takes a file; Up goes to the folder
above. Hidden entries are left out. It is the tool's own: the program
does not link a toolkit's file dialog here.
A GGUF source is one command,
oklm quantize. A checkpoint is one
command, oklm convert --type T, for the types that command
writes (F32, F16, BF16, Q8_0, Q4_0, Q4_1, Q5_0, Q5_1: every matrix in
the type, as --pure does), and two for the others or with
an importance matrix: oklm convert --type f16 to a file
named after the result (result-F16.gguf, kept), then
oklm quantize of that file.
The estimate is the command's own
--dry-run (convert.md), called
through the library from a thread of the window: the source's headers
are read, the type of each tensor is chosen as the command chooses it,
and the size of the file that would be written is added up. For the
two-step path the size of the F16 file is not the result's: the result's
is given once the F16 file exists (after a first run), and the window
says so until then. The source is read again a quarter of a second after
the fields stop changing.
The commands run as background processes of the
registry (server.md), of
kind convert, quantize or ppl:
detached, their output in a log file there. They go on when the window
is closed, and oklm serve --list shows them. The window
follows the log and reads the command's progress from the file its
--progress-json writes beside the logs (training.md): the bar
is the bytes written of the file's size, or the windows scored of the
text's, and the end line says whether the command succeeded
and, for oklm ppl, the perplexity. A command whose process
is gone without an end line failed; the log says why. The
commands of a button run one after the other, and the first that fails
ends the queue.
Options
| Option | What |
|---|---|
-m SOURCE |
the source: a GGUF file or a checkpoint's folder |
--out FILE.gguf |
the result |
--type T |
the type (Q4_K_M) |
--pure |
One type everywhere ticked |
--imatrix FILE |
the importance matrix |
--name NAME |
the name written in the file |
--text FILE.txt |
the text of the measure |
-c N |
the measure's window, tokens (512) |
--run |
start the conversion once the source is read |
--check |
start the measure once the result is there |
With --screenshot, the window is saved when the source
is described; with --run or --check, when the
commands are over.
Limits. The measure runs on the CPU. The importance matrix is not
computed here (oklm imatrix). On Windows the commands run
in the background through the registry as on Linux (since 2026-11-13);
Stop terminates the command, which cannot be asked to
end there. Run on 2026-11-13 with oklm-gui.exe under Wine
11 and Xvfb, not on Windows itself: SmolLM2-135M F16 requantized to
IQ3_M and stopped on the way (the bar moving, then the .tmp
file of 63 MB removed and no process left); the same file to Q4_K_M to
its end (105454144 bytes, the size announced), then Measure
perplexity on 6 kB of prose with 512-token windows:
Source: perplexity 10.139,
Result: perplexity 10.417 ... +2.74 % against the source
(oklm ppl on Linux gives 10.1391 for the source); the
unattended run of make gui-test. The file browser of
Browse... cut paths at / only: on Windows
it opened on the working directory and Up emptied it;
both are fixed (it opens on the folder of the file named,
Up stops at Z:\). From a Hugging Face
folder: models/smollm2-135m-hf to Q4_K_M in its two steps
(the F16 file of 270885696 bytes kept, the result of 105453888 bytes as
announced), and the fixture tests/fixtures/hf/tiny-llama to
Q8_0. Stop on a conversion to IQ3_M: the command ended
by itself (writing ... was stopped: nothing is left of it),
its partial file of 56 MB gone.
The model inspector
Draws a model as a map and says where each of its layers is and what
memory that takes: for a file, projected by the memory planner without
loading a weight; for a running server, as it really is, with the
server's figures. It is what oklm info -t and
oklm plan print, as a picture whose controls can be moved
to see what a context or a split costs before anything is loaded. It
also shows what happens inside the model while it runs: a prompt run
through a file's model on the CPU, or the token a server is decoding
(and the ones before it, which the server keeps), with what each block
wrote, where the attention of a block looked, which experts the routers
picked, what the output head would say after each block, and what the
output loses when a block's write or a head is taken out (what those figures
mean).
oklm-gui --tool inspect # choose the source in the window
oklm-gui --tool inspect -m model.gguf -c 8192 # a file, the placement projected at 8192 tokens
oklm-gui --tool inspect -m model.gguf --backend opencl --gpu-layers 12 --n-cpu-moe 6
oklm-gui --tool inspect -m model.gguf --ram 8192 # projected against 8 GiB of host memory
oklm-gui --tool inspect --url http://127.0.0.1:7430 [--key KEY] # the model a server holds, live
oklm-gui --tool inspect -m model.gguf --backend cpu --prompt "The capital of France is" --tokens 8 --layer 5
# run a prompt on the CPU: the blocks' writes, block 5's attention, the experts
oklm-gui --tool inspect -m model.gguf --prompt "The capital of France is" --step 0 --view lens # what the head would say after each block
oklm-gui --tool inspect -m model.gguf --prompt "The capital of France is" --step 0 --view effects # each block's write taken out in turn
oklm-gui --tool inspect -m model.gguf --prompt "What is the capital of France?" --chat # the prompt through the chat template
oklm-gui --tool inspect --url http://127.0.0.1:7430 --step 41 # a server's record 41, among those it keeps
Menu entry: OKLM Model Inspector.

Example: a file, and where its layers would go
- Start
oklm-gui --tool inspect -m models/granite-moe/granite-3.1-1b-a400m-instruct-Q4_K_M.gguf(or choose Model file, type the path and press Inspect). The status line saysits tensor table only, no weight is loaded. - The header: Granite 3.1 1B A400M, architecture
granitemoe, 1.33 B parameters, Q4_K_M, trained context 131072,24 layers, 32 experts a layer (8 used a token), 783.8 MiB, and in greenCompatibility: loadable. - The map: the embeddings, then a bar a block, almost all of each violet, the routed experts. With nothing pointed at, the panel under it sums the model up by role: routed experts 697.5 MiB (89.2 %), attention 42.0 MiB, embeddings and output 39.4 MiB.
- Click block 5 (
--select 5at start shows it as clicked). It gets a white frame, the panel readsBlock 5 10 tensors, 33.1 MiB, 53.5 M parameterswith its tensors by role and their types (routed experts: 3 tensors, 31.1 MiB (Q4_K x2, Q6_K x1)), where it is and its cache (Key/value cache: 8.0 MiB), and the tensor table lists its ten tensors: the picture above. - On the right, the placement on the CPU at 4096 tokens:
Host memory: 1.89 GiBof what is available (weights 1.53 GiB, cache 192.0 MiB, buffers 184.2 MiB) andVerdict: fits resident. - Choose
openclin Backend, 12 with Layers on the device and 18 with Experts in host memory (--backend opencl --gpu-layers 12 --n-cpu-moe 18at start). The planner asks the device its memory; nothing is loaded on it. Under the map, layers 12 to 23 turn green, and layers 12 to 17 are cut in two: on the device, their routed experts in host memory. The panel readsHost memory: 1.32 GiB,Device memory (opencl): 517.6 MiB of 18.32 GiBand, in amber,Verdict: fits with 12 of 24 layers on the device.

Example: a running server
A server holds the same model:
oklm serve -m models/granite-moe/granite-3.1-1b-a400m-instruct-Q4_K_M.gguf --backend cpu -c 2048 --parallel 2 --port 7430.
It has answered one request of 120 tokens.
- Start
oklm-gui --tool inspect --url http://127.0.0.1:7430(or choose Running server and pick the server in the list, which connects to it). - The map is the same, every layer blue: in host memory. The panel on
the right is now Server (measured):
Backend cpu,Context: 2048 tokens x 2 sequences,Cache blocks: 0 used, 512 free of 512. - Under them, the figures:
Requests: 1 answered, 0 of 2 generating,Tokens: 55 prompt, 0 cached, 120 generated,Last request (generate, length): 1392 mswithqueue 0.1, prompt 56, decode 1335 ms, the decode steps (mean 11.4, median 10.3, p90 11.8 ms), and the memory:Weights: 782.1 MiB host, 0 B device,Cache: 384.0 MiB host. They are read again every second.

Example: see which blocks and experts answer a prompt
- Start
oklm-gui --tool inspect -m models/granite-moe/granite-3.1-1b-a400m-instruct-Q4_K_M.gguf --backend cpu, typeThe capital of France isin the field under the file's and press Run (or Enter;--prompt "The capital of France is"at start does the same). The status line saysLoading granite-3.1-1b-a400m-instruct-Q4_K_M.gguf on the CPU: 1.73 GiB of host memory (111.97 GiB available)..., thenRan on the CPU, 1.73 GiB of host memory: 5 prompt tokens, 8 generated.The model wroteParis., two line feeds andThe capital of. - Right of Run:
Token 9 of 9: " of", the last token generated. The slider beside it, or Left and Right, step through the nine: the prompt's last token (is), then each generated one. - Over the map, a strip named
writes: for the token shown, two bars a block, orange what its attention added to the residual stream, violet what its experts added (green for a dense feed-forward part), against the largest of the blocks. Here block 23 stands far above the others (its experts wrote 26.75, its attention 15.43; the next is block 22's experts, 7.06): the last blocks write the most, because the stream has grown along the way (which is not the same as deciding the most: observe.md). - Under the map, on the right, Experts the routers
picked: a row a block, a column an expert, lit by how many of
the run's nine tokens the block's router sent to it; a dot marks the
eight picked for the token shown. Pointing at a cell reads it out:
Block 5, expert 8: picked for 7 of 9 tokens (78 %); for the token shown: rank 1, weight 0.254. The picture below. - Click block 5 in the map (or its row in the experts;
--layer 5at start). The panel becomes Block 5 reading for " of" (position 12): a row a head (16), a column a position, brighter where the head put more of its attention, the tokens' texts under the columns. Pointing at a cell givesHead 15 gives 78.6 % of its attention to " of" (position 12), the token itself. Under the rows, block 5's 32 experts: the eight picked lit by their weight. - Press Left until
Token 1 of 9: " is"(--step 0at start): the prompt's last token, whose attention has the five positions of the prompt. Its experts in block 5 are 8, 13, 24, 29, 20, 10, 15 and 3, expert 8 with weight 0.363. On the left, the block's details end withFor " is": attention wrote 0.5342, feed-forward 0.5042 (norms)andThe stream after the block has norm 2.046. The second picture.


Example: at which depth the answer appears, and which parts it needs
- Start
oklm-gui --tool inspect -m models/llama3.2-1b/Llama-3.2-1B-Instruct-Q4_K_M.gguf --prompt "The capital of France is" --tokens 4 --step 0and choose Lens in the list beside Activations (--view lensat start). The panel becomes What the output head reads after each block, for " is": a cell a block, with the three tokens the model's final norm and output head give for the residual stream as it is after that block, the likeliest first, brighter for a higher probability. Blocks 0 and 1 readisback (the token itself), blocks 2 to 10 read word pieces of no relation (acos,otope,olit), and from block 11 on the first token isParis, framed in white because it is the model's own answer: 71 % at block 11, 79 % at block 13, 61 % at the last. Pointing at a cell reads it out:After block 11 the output head reads " Paris" 70.6 %, " Brussels" 8.3 %, "Paris" 4.7 %. Clicking a cell selects the block, whose details on the left then end withThe output head reads " Paris" there (71 %). - Choose Effects (
--view effects). The worker evaluates the prompt again with one part zeroed at a time, 32 times for this model (each block's attention write, then its feed-forward write), and the bars come as they are computed: orange for a block's attention, green for its feed-forward part, as high as the next token's distribution moved from the plain run's (the Kullback-Leibler divergence, on a square-root scale). Block 0's and block 1's feed-forward parts stand far above the rest (8.4 and 6.4 nats: without themParisis gone), then block 14's (2.8); the attention writes move the output by less than 0.2, block 0's (1.3) and block 15's (0.27) apart. Pointing at a block reads both:Without block 11's attention: 0.0344 nats, " Paris" 61 % -> 59 %, likeliest " Paris" 59 %; without block 11's feed-forward part: 0.705 nats, " Paris" 61 % -> 19 %, likeliest " Paris" 19 %. - Click a block (its column in the chart, or the map;
--layer 11): the same for each of its heads, a bar a head, here 32 more evaluations. None of block 11's heads matters much alone: the output moves most without head 10, by 0.0147 nats. A second click on the block in the map lets go and the blocks' chart comes back.


What the two views say and do not say is in observe.md: a lens is the model's last step applied where it was not trained to read, and a part taken out is another model's output.
The same on a server while it answers. Start it with
--observe:
oklm serve -m models/granite-moe/granite-3.1-1b-a400m-instruct-Q4_K_M.gguf --backend cpu --observe,
then oklm-gui --tool inspect --url http://127.0.0.1:7430.
While a request is decoded the row under the address reads
Activations (live): sequence 0 decoded " to" at position 79, record 76,
the strip over the map moves with each token, and the panel shows the
experts picked over the answer being generated (the server's
/api/experts). Click a block: the server is asked for that
block's attention and the rows arrive with the next token it decodes. A
server started without --observe shows
Activations: this server does not record them (start it with oklm serve --observe).
and its experts alone.
Once the answer is given its tokens can be gone back through: the
server keeps its last records (server.md), and the slider at the
right of the row (or Left and Right) steps through the serials kept for
the sequence. The row then reads
Activations (kept): record 10 of 1 to 14, sequence 0 decoded " the" at position 13,
and the views show that record, with the attention rows of the blocks
that were asked for when it was made. At the right end the window
follows the server again. A record the server has dropped meanwhile
gives
That record is no longer kept by the server: the last one is shown.
A server started with --observe-lens 3 gives the
Lens view too; the Effects view is for
a file's run (a server's model is not changed).
The window
| Part | What |
|---|---|
| Model file, Running server | the source |
| Activations | shows the activation panel in place of the tensor table once there is something to show (a prompt was run, or a server's activations or experts were read); unchecked, the table stays |
| The view list | what the activation panel shows: Writes, attention (the block selected's attention, else the experts picked or what each block wrote), Lens or Effects (below) |
| A file: the prompt, Chat, Tokens, Run | a prompt for the file's model: completed as it is, or with
Chat checked (--chat) given as a user's
message to the model's chat template with the opening of the answer, as
a chat does (the status line then says
(the message in the model's chat template); a model without
template runs the text as it is and says so). Run (or
Enter in the field) asks the memory planner what loading the model on
the CPU takes at 512 tokens and refuses when that does not fit the host
memory
(Not run: the model takes 229.2 MiB of host memory on the CPU, 200.0 MiB is available.,
here with --ram 200); else the worker thread loads it,
reads the prompt, generates up to Tokens tokens (8; 1
to 64), greedy, stopping at an end-of-turn token, and records the
prompt's last token and each generated one. The model stays loaded for
the next prompt until another file is read |
| The token shown, its slider | Token 3 of 9: " Paris": which of the run's tokens the
views show. The slider or Left and Right (outside a text field) step
through them. On a server the row says what it decoded last, read four
times a second, and the slider steps back through the records the server
keeps for the sequence (--step S at start: the record of
serial S); at its right end the last record is followed as
it comes |
writes, over the map's bars |
for the token shown, what each block added to the residual stream: the norm of its attention's part (orange) and of its feed-forward part (green, violet for routed experts), as bar heights against the largest write of the blocks. A grey mark: a block that was not recorded (a device that could not be read back; a server on a device records its blocks as the CPU's: observe.md) |
| The activation panel, Lens | a cell a block, in columns read downwards: the three likeliest next
tokens the output head gives for the residual stream after the block,
brighter for a higher probability, the first one framed where it is the
model's own answer. A click on a cell selects the block; the block's
details on the left end with
The output head reads " Paris" there (71 %). On a server,
when it was started with --observe-lens |
| The activation panel, Effects | for a file's run: how far the next token's distribution moves (Kullback-Leibler divergence in nats, bar heights on a square-root scale) when a part is zeroed, the tokens up to the one shown evaluated again for each part by the worker. With no block selected, two bars a block (orange its attention write, green its feed-forward write); with one selected, a bar a head of the block. A grey mark: not a part of the block (Phi-2's attention, a head of a block without attention). The line reads out the part under the pointer; the status line counts the parts as they are computed (one evaluation of the prompt each: 2 x blocks, then one a head) |
| The activation panel, Writes, attention | with a block selected: its attention for the token shown, a row a head (the first 32), a column a position (the last 128), brighter for a larger share, the tokens' texts under them and the token itself at the right end; under it, for a mixture of experts, the block's experts, those picked lit by their weight. With no block selected: for a mixture of experts the map of the experts the routers picked (a row a block, a column an expert, lit by the share of the tokens of the run, or of the server's last answer, it was picked for; a dot for the token shown), where a click on a row selects the block; for a dense model a chart of what each block wrote, with the stream's norm as a line, where a click selects the block. The line under the title reads out the cell under the pointer |
| Tensor table | shows or hides the table at the bottom right |
| The status line | what the window is doing, and what the last reading said |
| A file: the list, the field, Inspect | the list offers the models installed with
oklm models pull; the field takes a GGUF file's path (any
shard of a split model) or an installed model's name, read by
Inspect or Enter. Only the tensor table is read
(oklm_model_info_json(), what
oklm info --json -t prints: convert.md): no weight is loaded,
whatever the file's size |
| A server: the list, the address, the key, Connect, Rescan | the list holds the servers answering on this machine (ports 7430 to
7480, found as the chat window's Connect window finds them,
Rescan to look again); the fields take another address
and a key, read by Connect or Enter. The model and the
placement come from GET /api/model?tensors=1 (server.md): where the layers really
are. A server whose model is not a GGUF file (a Hugging Face checkpoint)
has no layout to draw; its figures are still shown |
| The header | the model's name and file; its architecture, parameters, quantization, trained context, layers (and experts a layer for a mixture of experts), tokenizer and file size; the compatibility verdict with its notes, green when OKLM loads the file |
| The map | the embeddings, one bar a block, the head (and the tensors that are none of them, when there are any). A block's height is its bytes against the largest block's; the embeddings and the head take the full height and the width their bytes ask at that scale, so that areas compare. Each is split by role, a colour a role, with the legend above (only the roles the model has): embeddings, attention, dense FFN, shared experts, routed experts, routers, norms, multi-token prediction, other |
where, under the map |
where each part is: blue in host memory, green on a device (another
colour a device when the layers are spread over several). A layer whose
routed experts are not where the layer is (--n-cpu-moe) is
cut in two, its lower half the experts' |
KV, under it |
each layer's key/value cache, in the colour of where it is |
| The details, bottom left | the whole model by role (bytes and share) when nothing is pointed at; the part under the pointer, or the one clicked: its tensors by role with their element types and bytes, where it is with its bytes on either side, where its routed experts are, its cache |
| The tensor table | every tensor (name, type, shape, bytes, role), or those of the part clicked; a click on a heading sorts |
A click on a part keeps it (a white frame) and lists its tensors; a second click, or one beside the parts, lets go.
The placement of a file is projected by the memory
planner (oklm_mem_plan_compute(), what
oklm plan --json prints: memory-plan.md) each time a
control moves, on a thread of its own:
| Control | What |
|---|---|
| Backend | cpu, opencl or vulkan. A
device backend makes the planner ask the device its memory; nothing is
loaded on it |
| Context | tokens, powers of two up to the context the model was trained with
(-c) |
| Layers on the device | --gpu-layers: the last layers, with the head; at the
left end, as many as fit (the number the planner chose is then shown).
Not used on the CPU backend |
| Experts in host memory | --n-cpu-moe: the first layers whose routed experts stay
in host memory. For a mixture of experts on a device backend |
Under them: host memory needed against what is available, as a bar whose end is the headroom kept free, with the weights, the cache and the buffers; the same for the device against what it takes; and the planner's verdict (green: fits; amber: fits with some layers on the device; red: does not fit, the bar and its line red too).
The figures of a server take the panel's place, read
every second on the worker thread from GET /api/stats and
GET /api/profile: the backend, the context and the
sequences; the paged cache's blocks used and free, as a bar; tokens a
second; requests answered and generating; the queue and the mean wait;
prompt, cached and generated tokens; the last request's profile (queue,
prompt and decode times, its tokens) and the decode steps' mean, median
and ninth decile; the routed-expert cache and the device expert tier
when the server runs with one (slots, bytes, hits); the weights and the
cache on either side and the context's host memory. The placement is
read again every five seconds: it follows a resize of the context. What
a route does not give is not shown.
Options
| Option | What |
|---|---|
-m MODEL, --model MODEL |
a GGUF file or an installed model's name |
--url URL |
a running server |
--key KEY |
that server's API key |
--backend cpu|opencl|vulkan |
where the Backend list starts |
-c N |
where the Context slider starts |
--gpu-layers N |
where Layers on the device starts |
--n-cpu-moe N |
where Experts in host memory starts |
--ram MIB |
project against this much host memory, in MiB, instead of what this machine has available |
--select embeddings|BLOCK |
the part shown as clicked when the window opens |
--prompt TEXT |
with -m: a prompt run through the model once the file
is read |
--chat |
the prompt is a user's message, run through the model's chat template |
--view writes|lens|effects |
what the activation panel shows when the window opens |
--tokens N |
tokens generated after it (8; 64 at most) |
--step N |
which of the run's tokens is shown: 0 the prompt's last, 1 the first generated... (default: the last) |
--layer BLOCK |
the block selected, whose attention is shown (the same as
--select BLOCK); on a server started with
--observe, the block whose attention it is asked for |
With --screenshot FILE.png the window is saved once what
was asked for is shown (a file's map and its projection; a server's map
and two readings of its figures), and a line on the error stream says
what it shows:
oklm-gui: inspect: 24 blocks, 242 tensors, 820107736 bytes; placement: fits resident
(make gui-test checks that line on its test model). With
--prompt the window is saved once the run is drawn and the
line goes on:
; activations: token 1 of 9 " is" at position 4, 24 of 24 blocks recorded, block 5's attention, 16 heads x 5 positions
(checked too, on the test model with --layer 1). With
--view lens it ends with
the lens of 16 blocks, 3 tokens a block: " Paris" read first from block 11 on,
and with --view effects, once every part was taken out,
with
32 parts taken out one by one, the output moves most without block 0's feed-forward part (8.36 nats)
or, with a block selected,
block 11's 32 heads taken out one by one, the output moves most without head 10 (0.0147 nats)
(both checked on the test model). On a server it waits for two readings
of its activations, and with --step S for the record of
that serial:
record 10 of 1 to 28 kept, " the" at position 13.
Limits of the activation views. The run of a file is on the CPU only,
whatever Backend says (that list projects a placement,
it loads nothing), in a context of 512 tokens: a prompt and its tokens
must fit. It is greedy, and a raw completion unless
Chat is checked: the message then goes through the
model's chat template alone, without a system message beyond the
template's own. Attention rows are kept for the last 128 positions and
the first 32 heads of each block, and not at all when a very large
model's rows do not fit the recording's memory; the norms, the experts
and the lens are always there. The Effects view
evaluates the tokens up to the one shown once for each part, on the CPU:
twice the blocks, then the heads of a block when one is selected; for
Llama 3.2 1B and a prompt of 6 tokens that is 32 then 32 evaluations and
a few seconds, for a chat prompt of 40 tokens or a model of many blocks
it is minutes (the bars come as they are computed, and another token or
block asked for stops the scan under way). It zeroes the parts; the mean
in their place is the command's (oklm run --ablate-mean). A
server records the attention of four blocks at once at most and of the
last 256 positions (fewer on a server of many sequences and heads); the
blocks it runs on a device are recorded as the others. What the views
say and do not say (a large write is not an important one, attention is
not an explanation, a lens is not a decision): observe.md.
Other limits. No native file dialog (the path is typed or pasted).
The pictures of the placement and of the server above were taken before
the row of the prompt was added under the source's, and those of the
experts and of the attention before the view list and
Chat. On Windows the window is the same. Run on
2026-11-13 with oklm-gui.exe under Wine 11 and Xvfb, not on
Windows itself, before the activation views were added (they are
compiled for Windows, not run there): the tiny model's file
(inspect: 2 blocks, 20 tensors ... placement: fits resident,
the line make gui-test checks) and a server started with
oklm.exe serve on SmolLM2-135M (--url: 30
blocks, 272 tensors, the panel Server (measured) with its
figures read every second).
The training window
A form for each kind of run, the oklm command the form
makes, and that command followed while it runs: the loss as a curve, the
checkpoints, the samples, the log. The window computes nothing itself:
Start runs oklm train,
oklm rl, oklm refusals or
oklm abliterate as a background process through the
registry (server.md) with
--progress-json on a file beside the run's output, and the
window reads that file's lines as they are written (training.md). The run
goes on when the window is closed, and is found again when it is
opened.
oklm-gui --tool train
oklm-gui --tool train --mode lora --set model=model.gguf --set data=text.txt --set out=build/lora/run
oklm-gui --tool train --mode scratch --set data=text.txt --set preset="Tiny: 2 layers, 64 wide" --start --screenshot run.png
Menu entry: OKLM Training.

Example: a small model from scratch
- Start
oklm-gui --tool train. The From scratch tab is shown. - Text file:
text/shakespeare.txt(12 kB), typed or chosen with.... Output prefix:out/tiny/tiny. - Size:
Tiny: 2 layers, 64 wide. It fills Vocabulary 512, Layers 2, Width 64, Heads 2, Context 64. - Steps 300, Batch 8,
Warm-up 20, Validate every 25,
Save every 100, Sample every 100,
Backend
cpu. - The box at the top right shows the command:
oklm train -f text/shakespeare.txt --out out/tiny/tiny --arch gpt2 --vocab 512 --layers 2 --embd 64 --heads 2 --ctx 64 --steps 300 --batch 8 --lr 1e-3 --warmup 20 --eval-every 25 --save-every 100 --sample-every 100 --backend cpu --progress-json out/tiny/tiny-progress.jsonl. - Press Start. The status line counts the steps with the two losses and the rate, the curve grows, and a checkpoint is listed every 100 steps.
- Three seconds later:
Ended step 300 / 300 loss 3.9159 val 4.3831 49972 tokens/s. The training loss went from 6.4 to 3.9; the validation loss flattens near 4.4, the text being small. Checkpoints hastiny-000100.gguf,tiny-000200.ggufandtiny-000300.gguf(560 800 bytes each), the last chosen; Samples has the texts generated, and the last line saysDone: 3 checkpoints. Choose one in the list for the buttons below.
Open a chat then starts the chat window on the checkpoint chosen.
The picture was taken with the form filled and started from the command line, which is the same run:
oklm-gui --tool train --mode scratch --set data=text/shakespeare.txt --set out=out/tiny/tiny \
--set "preset=Tiny: 2 layers, 64 wide" --set steps=300 --set batch=8 --set warmup=20 \
--set eval-every=25 --set save-every=100 --set sample-every=100 --set backend=cpu --start
The LoRA tab has the model in place of the size: here SmolLM2-135M, a rank of 4 and an alpha of 8, eight steps of one 64-token sequence, and the command they make.

Example: the Advanced tab
The three stages on the smallest setup: SmolLM2-135M and two files of
four prompts each, text/declined.txt (requests a model may
decline) and text/answered.txt (plain requests).
- Choose the Advanced tab
(
--mode advanced). - Model:
models/smollm2-135m/SmolLM2-135M-Instruct-Q8_0.gguf. Prompts the model declines:text/declined.txt. Prompts the model answers:text/answered.txt. Model written:out/abl/smol-abl.gguf. - Stages:
Measure, change the model, measure again. Answer tokens: 24. Backend:cpu. Layer and Scale are left empty (auto, 1). - The command box holds three commands,
oklm refusals,oklm abliterate,oklm refusalsagain. - Press Start (
--start). The stages run one after the other, a few seconds in all here. - At the end the plot reads: The model, prompts
declined
25.0 %,1 of 4; The model written,0.0 %,0 of 4,KL 0.123; and a bar a layer for the separation of the two sets,layer 0 to 29, highest 4.15, the bar of layer 15 in another colour:removed: layer 15. The list has each prompt as it was judged (after 4/4 answered: ...) and the file written; Answers judged has each answer; the log ends withfirst-token KL divergence from the reference: 0.1229.out/abl/smol-abl.ggufis 179 025 760 bytes.
Open a chat then starts the chat window on the model written.
Four prompts say nothing about a model; a measure needs sets of hundreds, and a held-out file in Prompts measured (training.md).

The tabs
One a kind of run; the fields they share keep what was typed, and take the tab's own values where nothing was.
| Tab | Command | Its fields |
|---|---|---|
| From scratch | oklm train -f |
Text file; Size (Tiny: 2 layers,
64 wide; Nano: 6 layers, 256 wide; Small: 8 layers, 512 wide; Custom: it
fills the five fields after it, which can then be changed);
Architecture (gpt2, llama,
qwen2, qwen3); Vocabulary,
Layers, Width, Heads,
Context |
| Fine-tune | oklm train -m |
Model, Installed models, Text file, Sequence |
| LoRA | oklm train -m --lora |
the same, with LoRA rank, LoRA
alpha and Targets (q,k,v,o when
empty) |
| Chat data | oklm train --chat-data |
Model, Conversations (JSONL), Sequence, and a LoRA rank (empty: every weight is trained) |
| RL | oklm rl |
Model, Tasks (JSONL), Tasks a step, Group (completions a task), Answer tokens, Temperature |
| Advanced | oklm refusals, oklm abliterate |
Model, Prompts the model declines, Prompts the model answers, Model written, Stages, Layer, Scale, Answer tokens, Prompts judged, Prompts measured |
| Shared field | Option | What |
|---|---|---|
| Output prefix | --out |
checkpoints are PREFIX-STEP.gguf; the progress file is
PREFIX-progress.jsonl and the log
PREFIX-run.log. The directories are made when missing.
oklm-train/run under the home directory until another is
typed |
| Steps | --steps |
updates of the weights |
| Batch | --batch |
sequences a batch |
| Learning rate | --lr |
the highest rate, reached after the warm-up, then lowered along a cosine |
| Warm-up | --warmup |
steps over which the rate rises from zero |
| Validate every | --eval-every |
steps between two measures on the held-out data (the second curve); 0 for none |
| Save every | --save-every |
steps between two checkpoints; one is always written at the end |
| Sample every | --sample-every |
steps between two texts generated by the model as it is |
| Backend | --backend |
auto, cpu, opencl,
rocm |
| More options | added to the command as a shell would split it (in the Advanced tab:
to oklm abliterate) |
A model is a file, typed, chosen in the system's file dialog (the
... buttons), or taken from Installed
models, the catalogue's models that are on the machine (models.md). An empty field leaves its option out,
so the command's own default applies; the greyed text names it. Each
field's tooltip says what the option does.
The run
| Control | What |
|---|---|
| The command box | the command line Start will run, wrapped (the wheel moves through it); in the Advanced tab one command a stage |
| Copy | the command on the clipboard, as one line a terminal takes |
| Start | runs the command in the background |
| Stop | asks the run to end (SIGTERM; on Windows the registry's stop event):
it finishes the step it is in, writes the checkpoint of that step, which
--resume goes on from, and says
stopped on request at step N. A run that has not ended
after four seconds is ended by the system (SIGKILL), the checkpoints
written so far kept |
| The status line | the step and the total, the last training and validation losses, the tokens trained on a second, the time elapsed and the time left as the command estimates it |
| The curve | the training loss at each step and the validation loss where it was measured (the mean reward and the held-out accuracy for RL), over the steps made so far |
| Checkpoints | each file as it is written, the newest chosen; the buttons below act on the one chosen |
| Samples | the texts generated at the steps Sample every names |
| Log | the end of what the command prints, its errors too |
| Open a chat | the chat window on the checkpoint chosen, the merged model or the
model written, a process of its own
(oklm-gui --tool chat -m FILE --backend ...). A LoRA
checkpoint is an adapter, not a model: the button is on the merged model
once there is one |
| Merge adapter | after a LoRA run:
oklm lora-merge MODEL ADAPTER ADAPTER-merged.gguf in the
background, its output in the log (training.md) |
| Export adapter... | after a LoRA run: copies the adapter to the file the system's dialog
gives. There is no export command because none is needed: a LoRA
checkpoint is already an adapter file in llama.cpp's format, which
oklm run --lora, oklm serve --lora-module and
llama.cpp's --lora take |
| The last line | what the last press did, and how the run ended |
A run found again. Opened without an option, the
window looks in the registry for a run going on
(oklm train, rl, refusals or
abliterate with --progress-json on a file),
the newest first. It reads the progress file's path from the registered
command line, fills the form from that command line (a training run's:
what no field holds goes to More options), and shows
the run from its first line: the whole curve, the checkpoints, the
samples.
Advanced: the refusal direction. The tab runs the three stages of training.md one after the other, or one alone (Stages):
oklm refusals -m MODEL --prompts FILE: the share of the prompts the model declines;oklm abliterate -m MODEL --harmful DECLINED --harmless ANSWERED --out OUT: the direction that separates the two sets of prompts, found at each layer and removed from the weights at the layer chosen (Layer,autoby default; Scale for less than the whole of it);oklm refusals -m OUT --prompts FILE --ref MODEL --harmless ANSWERED: the share again on the model written, and the mean first-token KL divergence from the model on the prompts it answers, which says how far the change moved the rest of its behaviour.
The prompts measured are the declined set unless Prompts
measured names another file (a held-out part of it is the
better measure). The plot shows the two shares side by side, each with
its count of prompts, the KL divergence under the second, and each
layer's separation as a bar, the layer removed in another colour. The
list holds the per-layer figures, each prompt as it is judged and the
files written; the answers judged are beside it. Each stage writes its
own progress file (OUT.before.jsonl,
OUT.abliterate.jsonl, OUT.after.jsonl), so a
stage run alone later keeps the others' figures on screen.
Options
| Option | What |
|---|---|
--mode scratch|finetune|lora|chat|rl|advanced |
the tab shown at start |
--set FIELD=VALUE |
a field's value, as many times as needed. The names:
model, installed, data,
chat-data, tasks, harmful,
harmless, out, model-out,
preset, arch, vocab,
layers, embd, heads,
ctx, rank, alpha,
targets, steps, batch,
seq, lr, warmup,
rl-tasks, group, max-new,
temp, eval-every, save-every,
sample-every, stages, layer,
scale, answer-tokens, limit,
measure, backend, more
(oklm-gui --tool train -h lists them) |
--start |
start the form's command at once; with --screenshot,
the window is saved when the run has ended |
--oklm PROGRAM |
the oklm the commands run, instead of the one beside
oklm-gui (else the one on PATH) |
--press PRESS |
a test aid, given once for each press: a button pressed without a
person as soon as it can be, each in its turn. stop:
Stop, once a run is going on (the one
--start started, or one the window found), done when the
run has ended. merge: Merge adapter, once
a LoRA run has ended, done when the merged model is written.
export: Export adapter..., which opens the
system's dialog, done when the dialog is closed.
export:FILE: the adapter copied to FILE, the dialog's
answer without the dialog. chat: Open a
chat. With --screenshot the window is saved after
the last one (The
presses) |
Limits. One run is followed at a time. The window does not choose the
data: a text, conversations or tasks are files made beforehand (training.md). On Windows a run starts in the
background through the registry as on Linux (since 2026-11-13), and
Stop asks it to end through the registry's event, as
SIGTERM does on POSIX: the run writes the checkpoint of the step it
reached. Run on 2026-11-13 with oklm-gui.exe under Wine 11
and Xvfb, not on Windows itself, from scratch on 300 kB of text with a
model of two layers, 64 wide: a run of 200 steps to its end (the two
curves, four checkpoints, Ended step 200 / 200); a run of
200000 steps left by its window, which went on, was shown by a window
opened afterwards
(Found a run going on (process 380, oklm train): shown here),
and stopped by Stop (Stopped step 3100,
its checkpoints kept); the ... button, whose dialog is
Wine's and gave the file back. A checkpoint's row named the whole path
and empty parentheses when the path was written with backslashes: fixed.
The other tabs, each run to its end: Fine-tune,
LoRA and RL on the tiny model of the
server test (20 steps with a checkpoint every 10; four steps on twelve
tasks), then Merge adapter
(lora-000020-merged.gguf), Export
adapter... through the system's dialog (a copy of the adapter
under the name typed) and Open a chat on the merged
model; Chat data and Advanced with
SmolLM2-135M, which has the chat template they need (four steps on eight
conversations, an adapter of 1859744 bytes; the three stages,
abl.gguf of 179025760 bytes, refusals: 0 of 4,
first-token KL divergence from the reference: 0.1233).
Stop now asks the run to end by itself there as SIGTERM
does elsewhere: it wrote the checkpoint of the step it had reached (The run).
The sessions browser
Shows the agent sessions of a project folder: the transcripts
oklm agent keeps in its .oklm/sessions (agent-sessions.md). It is for
finding a session by what was said in it, reading it with its tool
calls, and taking it up again, where oklm agent sessions
lists and exports. It reads the sessions through the harness
(oklm/harness/session.h), as that command does, and writes
nothing to them.
oklm-gui --tool sessions # the folder opened last, else the current one
oklm-gui --tool sessions --project DIR
oklm-gui --tool sessions --project DIR --search TEXT --session ID
Menu entry: OKLM Sessions.

Example: find a session, read it, export it
The folder project holds four sessions made with
oklm agent -p (three on Qwen3-0.6B with the Read and Grep
tools, one on SmolLM2-135M), and an agent started there with
oklm agent --listen is running a fifth.
- Start
oklm-gui --tool sessions --project project. The list has the five, the most recent first, with the turns, the model and the size. The running one is tinted, and its Live column readslive :7481, the agent's port. - Click the row
Read notes.txt and te.... On the right: its title, its id (20261113-005122-46e0),Qwen3-0.6B-Q8_0,1 turn, 3 messages, then the transcript, the user's message and the model's under headings with their time. Resume and Delete are enabled, Attach is not: the session is not live. - Type
lighthousein the search field. A moment later the list keeps3 of 5 sessions, a Hits column counts the lines found in each (1, 13 and 18), and the lines of the session read are listed under the list, each with the role it comes from. - Click the third line. The transcript goes to it, the word selected,
and the counter reads
line 3 of 18 found; Previous and Next walk the lines. - Press Export .md. With
zenityorkdialoga dialog asks where; without one, as here, the line at the bottom saysNo file dialog here (zenity or kdialog): written in the project folder. Markdown written, 6457 bytes:and the file,20261113-005122-46e0.mdin the folder. - To take the session up again with another model, type
-m qwen3-0.6b --backend cpuin the options field: they are added to the command beside it,cd FOLDER && oklm agent --resume 20261113-005122-46e0 .... Copy puts it on the clipboard (The command is in the clipboard.); Resume starts it in a terminal. - Click the live row: its heading ends with
LIVE on port 7481, Attach is enabled, Resume and Delete are greyed.

The controls
The folder (the top row).
| Control | What |
|---|---|
| Folder | the project's path; Enter or Open lists it. Without
--project the window opens on the folder opened last, else
on the current one |
| Browse... | the system's folder dialog (zenity, else
kdialog, started as a program; on Windows the system's
own). Without either the line at the bottom says so and the path is
typed |
| Open | lists the folder typed |
| The list beside it | the folders opened before, the latest first, twelve at most, kept
with the agent options below in
$XDG_CONFIG_HOME/oklm/gui-sessions.json
(~/.config/oklm, %APPDATA%\oklm on
Windows) |
| Refresh | lists again and looks for running agents |
The list (left).
| Control | What |
|---|---|
| The search field | keeps the sessions whose messages hold the text, whatever the case of its ASCII letters: the user's and the model's texts, the tools' results, the calls' names and arguments; the agent's reminders are left out. Searched a moment after the typing stops (or at Enter), by a thread of its own |
| The list | a row a session, the most recent first: when it was last written,
its title (the name /rename gave it, else the first line of
its first request), the user's turns, the model it was started with, its
transcript's size in KiB. A click reads the session; a click on a
heading sorts by that column, a second one the other way |
| Live | shown while an agent runs a session: a session an agent started with
--listen is running now is tinted and the column names the
agent's port (agent-loop.md).
They are looked for every two seconds, and the list is made again when
one starts, ends or writes (every four seconds at most) |
| Hits | shown while a text is searched: the lines found in each session |
| Lines found in the session read | those lines, with the role they come from; a click on one brings the transcript to it with the text selected |
Reading (right).
| Control | What |
|---|---|
| The heading | the session's title, id, model, start and counts |
| The transcript | the user's messages and the model's under headings with their time,
the conversation from before a compaction included. A tool call, a
tool's result, the model's reasoning and a reminder of the agent's are
one quoted line each (call Read {"file_path":...},
result Read 12 lines ...). Text is selected by dragging and
copied with Ctrl+C. A session being written is read again as it grows,
the view staying where it was |
| Tools in full | shows them as blocks instead (300 lines or 32 KiB of each result; the exports have them whole). A line found that sits in a folded result is reached once the box is ticked |
| Previous, Next | the line found before, the next one; the counter beside them says which |
What is done with the session (under the transcript).
| Control | What |
|---|---|
| Export .md, Export .json | the session written to a file chosen with the system's dialog, which
opens on ID.md or ID.json in the project
folder; without a dialog program the file is written there and the line
says so. Markdown is for reading (the requests, the answers, the calls,
the first lines of each result, the reminders left out); JSON holds
every message whole, for a program (agent-sessions.md) |
| Resume | a terminal started in the folder on
oklm agent --resume ID, with the options of the field under
it. The terminal is $TERMINAL (-e), then
x-terminal-emulator, gnome-terminal,
konsole, xfce4-terminal, kitty,
alacritty, foot, wezterm,
mate-terminal, lxterminal, xterm,
each given the command its own way; it stays open when oklm
ends with an error. On Windows a console of its own, started in the
folder on cmd.exe /k "oklm.exe agent --resume ID OPTIONS":
it stays when the agent ends, with what it printed |
| The options field | options added to oklm agent when the session is
resumed: -m MODEL or --url URL above all
(empty, the agent takes the model of its settings, else the default
one) |
| The command, Copy | the command that resumes the session, for a terminal of one's own: what is left when no terminal is found |
| Attach | a live session: the chat window started on the agent that runs it,
oklm-gui --tool chat --agent PORT, as the Connect window
attaches (Connecting). Resume
and Delete are off for a live session: two processes
would write one transcript |
| Delete | after a confirmation in a window of its own (Delete it, Keep it), the transcript, the copies of the files the session changed and their git references are removed |
| The last line | what the last action did |
Options
| Option | What |
|---|---|
--project DIR |
the project folder (else the folder opened last, else the current one) |
--search TEXT |
the text searched at start |
--session ID |
the session read at start |
--press PRESS |
a test aid, given once for each press, done in turn once the list is
shown. select:ID: a click on a session's row;
select:live: on the row of the first session an agent runs
now. Then, on the selected session: attach, which waits for
the session's agent to be known; resume;
export, Export .md, which opens the
system's dialog, done when it is closed; export:FILE, the
session written to FILE as Markdown with no dialog; delete,
and its question's answers yes (Delete it)
and no (Keep it). With
--screenshot the window is saved after the last one (The presses) |
Limits. One folder at a time; the sessions of other folders are
reached through the list of recent folders. The window reads and removes
sessions, it does not edit them. On Windows the window is the same; the
dialogs are the system's (Browse... a tree of folders,
the exports a file to save, which opens in the project folder on
ID.md or ID.json), the command of the field
and of Copy is written for cmd.exe
(cd /d "FOLDER" && "...\oklm.exe" agent --resume ID),
and the agents listening are found by their files under
%LOCALAPPDATA%\oklm\agents. Run on 2026-11-13 with
oklm-gui.exe under Wine 11 and Xvfb, not on Windows itself:
sessions kept by oklm.exe agent -p listed, searched (a word
found in one session of four), read, written as Markdown and as JSON (to
the project folder before the dialogs were there, then to a name typed
in Wine's dialog); a session marked live while
oklm.exe agent --listen ran it, and Attach
opening the chat window on it (a message answered there);
Resume opening Wine's console with the agent in it, at
its question about the folder's trust. Resume started
nothing under Wine as it was first written (cmd /c start,
every word between quotes): it now starts the console itself.
Export .md through the system's dialog wrote
ID.md in the project, the name proposed;
Delete asked in its window, kept the session at
Keep it and removed its transcript at Delete
it, the other session's left. Two sessions started in the same
second are listed in the order the folder gives them.
The retrieval manager
One index at a time, and what oklm rag does with it (rag.md), from a window: build or update it from
folders, see which of its sources are stale, search it, score its
settings on a judged set, and attach it to a chat window or a server.
The window does nothing of retrieval itself: each action runs a command
and reads the JSON it writes (rag.md).
oklm-gui --tool rag
oklm-gui --tool rag --index docs.gguf
oklm-gui --tool rag --index docs.gguf --tab search --query "how is the cache quantized"
oklm-gui --tool rag --index docs.gguf --model bge-small-en-v1.5 # the Build tab, with that embedding model chosen
Menu entry: OKLM Retrieval.

Example: build an index, see it go stale, update it
docs/ holds seven Markdown files, 433 KiB together.
- Start
oklm-gui --tool rag --index docs.gguf. The file does not exist yet: the line at the bottom saysNo index at that path yet: name what to index on the Build tab, and Build writes it there. - On the Build tab, type
docsin Folders and files to index. - In Embedding model, choose
bge-small-en-v1.5 F32. Query instruction fills in with the text that model expects (Represent this sentence for searching relevant passages:). The rest stays: chunkerauto, vectorsf16, devicecpu, Keywords (BM25) ticked. - Press Build or update. The bar reads the files,
then counts the chunks embedded
(
embedding: 320 of 717 chunks, 26 a second); under it7 of 7 files: 7 added, 0 updated, 0 unchanged, 0 removed, 0 skippedand a line a file with its chunks: the picture above. - At the end:
Index written: 7 sources, 717 chunks (717 embedded by this build), 1 MiB, in 28.5 s., and the line under the index's path sums it up:7 sources, 717 chunks, 1 MiB; encoder bge-small-en-v1.5 (384 dimensions) ... 0 stale. - Now change the folder: add a paragraph to
docs/models.md, deletedocs/build.md, copy a new filedocs/agent-loop.mdin. - On the Sources tab, press Check
again. The row of
models.mdis amber,changed (stale), that ofbuild.mdred,missing (stale), and the line beside the buttons says1 changed and 1 missing of 7: the index is stale.A new file is not a source yet, so it is not listed. - Press Update the index. The build runs with the
index's own settings:
8 of 8 files: 1 added, 1 updated, 5 unchanged, 1 removed, 0 skipped, thenIndex written: 7 sources, 705 chunks (75 embedded by this build), 1 MiB, in 2.6 s.Only the new and the changed file were embedded.

Example: search and attach
- Start
oklm-gui --tool rag --index docs.gguf --tab search --query "How do I change the context of a running server?"(or type the question on the Search tab and press Enter). - The line under the controls says what was done:
5 passages, hybrid search, fusion weighted; candidates: 50 by the vectors, 50 by the keywords. 26.9 ms: the encoder loaded in 13, the question embedded in 12, searched in 0.1.The list has the five passages, the first fromserver.md, lines 629 to 639, score 0.5979, found by both retrievers. - Tick Rerank and press Search. The
reranker reorders the candidates (
reranked in 873, of 903.7 ms): the first passage is nowserver.mdlines 897 to 909,Server > The dashboard, the one that namesPOST /api/context. - Click a row: the passage is shown under the list with where it is
from, each score it has
(
dense 0.6547 (rank 8); sparse 13.33 (rank 5); fused 0.567 (rank 3); reranker 0.05251) and its text, Markdown rendered. - On the Attach tab, the line for a server follows
the Search tab's settings:
oklm serve -m smollm2-135m --rag .../docs.gguf --rag-k 5 --rag-rerank .../ms-marco-minilm-l6-v2/ms-marco-MiniLM-L6-v2-F16.gguf, with the first installed chat model. Copy the line puts it on the clipboard. - Choose
qwen3-0.6bin the list (the line now names it) and press Open a chat window: a chat window serving Qwen3-0.6B with the index opens. Asked the same question there, the model reasons over the sources it was given and answers from the third (338 tokens): the picture in A retrieval index.


The index
| Control | What |
|---|---|
| Index | the index file's path; Enter or Open reads it. A path with no file yet is the index Build will write |
| The list beside it | the indexes used before, kept in gui-rag.json of the
settings directory ($XDG_CONFIG_HOME/oklm) with the folders
each was built from |
| Browse... | an existing index chosen with the system's file dialog |
| Open | reads the index: its figures, and each source compared with its file
(oklm rag info --json --sources) |
| The line under it | the open index: sources, chunks, size, encoder, when it was written, how many sources are stale |
| The line at the bottom | what the last action did |
The window opens on the last index used, or on
--index FILE. --model NAME chooses the
embedding model of the Build tab and shows that tab (unless
--tab names another): an installed embedding model of the
catalogue by its name, hash, or a model file, which goes in
the field under the chooser. It is what the chat window's Open
in Retrieval starts the manager with. A name that is no
installed embedding model is said
(--model qwen3-0.6b: it is not an embedding model) and the
choice stays as it was.
The tabs
Build.
| Control | What |
|---|---|
| Folders and files to index | one a line, typed, or added with Add a folder... and Add files.... Every file under a folder is read (text, Markdown, code, JSON, HTML, DOCX, PDF). Empty, an update reads again the files the index holds; a file no longer named is dropped by an update |
| Embedding model | the index's own (the hash encoder for a new index),
hash:384 (hashed words, no model), an installed embedding
model of the catalogue (models.md), or a file
named in the field under it (... chooses it). An existing
index is updated with its own; another one needs Rebuild
everything |
| Query instruction, Passage instruction | the text the model expects before a question and before a passage
(--query-prefix, --passage-prefix); filled in
for the catalogue's bge and nomic models |
| Chunker | auto (by functions for source code, by headings and
paragraphs for the rest), fixed, recursive,
code, semantic |
| Chunk size | target tokens a passage (--chunk-size, 256) |
| Vectors | f16, f32 or q8_0: how the
vectors are stored |
| Device | the backend (cpu, auto,
opencl, vulkan), also used by the search, the
evaluation and the chat window |
| Keywords (BM25) | keeps the words of each passage too, for the sparse and hybrid searches |
| Rebuild everything | ignores the index that is there and embeds every file again
(--rebuild) |
| Build or update | starts oklm rag index ... --progress-json FILE as a
background process and follows the file: new and changed files are
embedded, unchanged ones keep their vectors |
| Stop | ends the process; the index is then as it was |
| The bar and the lines under it | the files read, then the chunks embedded, with the rate; the current file; the counts of files added, updated, unchanged, removed and skipped; then the result (sources, chunks, chunks embedded, size, time) or the error |
| The list | a line a file: what became of it, its chunks, its path |
Sources.
| Control | What |
|---|---|
| The four lines | the encoder and its instructions; the model file; the chunker, the vectors, the postings, the graph, the captions; the directory all the sources share |
| Check again | reads the index and compares each source with its file again |
| Update the index | enabled when something is stale: runs the build with the index's own chunker, vectors and postings, so that only the changed files are embedded; missing ones are dropped, new ones of the folders named on the Build tab added |
| The table | a row a source: its state, chunks, size, the file's date when
indexed, what it was read as, its path. changed rows are
amber and missing rows red: the index is stale. A heading
sorts by its column |
Search.
| Control | What |
|---|---|
| The question, Search | Enter or the button runs oklm rag search on the open
index |
| Mode | the index's default (hybrid when it has keyword postings),
hybrid, dense (the vectors),
sparse (the keywords) |
| Passages | 1 to 20 (5): how many are returned (-k) |
| Rerank | reorders the candidates with the first installed reranker of the catalogue; without one the tooltip says which to install |
| The line under them | what was searched, the candidates of each retriever and the time of each stage |
| The list | a row a passage: rank, score, the retrievers that found it, source, lines, heading. Scores of different modes do not compare |
| The passage | the row chosen: where it is from, each score it has, and its text (Markdown rendered; other sources as code) |
Evaluate.
| Control | What |
|---|---|
| Judged set, Browse... | a directory in the BEIR layout: corpus.jsonl,
queries.jsonl, qrels/test.tsv (rag.md) |
| dense, sparse, hybrid, Rerank | the modes scored, and whether with the reranker |
| Run the evaluation, Stop | oklm rag eval DIR --json as a background process; ends
it |
| The table | a row a configuration: nDCG@10, MRR@10, recall at 1, 5, 10 and 100, precision at 5 and 10, the median and 95th percentile time of a query, the candidates reranked, the queries and the documents |
The evaluation scores the retrieval settings, not the open index's
documents: the set's own corpus is indexed, once, in the window's cache
directory (oklm/gui-rag), with the open index's encoder and
instructions, or with the encoder chosen on the Build tab.
Attach.
| Control | What |
|---|---|
| The model list | the installed chat model that answers; it also names the model of the server's line |
| Open a chat window | starts oklm-gui --tool chat -m MODEL --rag INDEX: a
window of its own whose answers come from the index (A retrieval index) |
| The line, Copy the line | the oklm serve -m MODEL --rag INDEX line that attaches
the index to a server, with the Search tab's mode, passages and reranker
(--rag-mode, --rag-k,
--rag-rerank); the button puts it on the clipboard |
Options
| Option | What |
|---|---|
--index INDEX.gguf |
the index opened at start (else the last one used) |
--tab build|sources|search|evaluate|attach |
the tab shown |
--query TEXT |
a search made once the index is read |
--model NAME|FILE |
the embedding model chosen on the Build tab, which is then the tab
shown: an installed catalogue model's name, hash, or a
model file |
--press PRESS |
a test aid, given once for each press, done in turn.
sources:PATH: a folder or a file typed in the Build tab's
field of what is indexed, a line more each time. build:
Build or update, done when the build has ended.
eval:DIR: the Evaluate tab shown, the judged set's
directory typed and Run the evaluation pressed, done
when it has ended. With --screenshot the window is saved
after the last one (The
presses) |
A build and an evaluation are processes of the registry (server.md, kinds
rag-index and rag-eval): they go on when the
window is closed, and their progress file and log are in the cache
directory until they end. A window opened meanwhile does not take a
running build back; Check again shows the index once it
is written. rag info and rag search run on a
thread of the window, through the command's own function, so the window
never waits on them. After rag info, the window's working
directory is the one the index was built from, as an index made from a
terminal records its sources and its model relative to it.
Limits. An index with a graph or with figure captions is shown and
searched, but not built or updated here: those need a chat or a vision
model (oklm rag index --graph --gen-model,
--caption-model). The advanced chunker settings (overlap,
smallest and largest chunk) are kept as the index has them and not
offered. The reranker is the first installed reranker of the catalogue.
On Windows a build and an evaluation are background processes of the
registry as on Linux (since 2026-11-13), and Stop
terminates them. Run on 2026-11-13 with oklm-gui.exe under
Wine 11 and Xvfb, not on Windows itself: an index built from a folder of
four files with hash:384 (277 chunks, the rows of the files
as they were read); Sources, then one file changed and
one removed and Check again
(1 changed and 1 missing of 4: the index is stale, the two
rows coloured), Update the index (3 sources, 253
chunks); a search (the changed file first for its sentence); another
index built with the installed bge-small-en-v1.5 (265
chunks at 13 a second, 19.8 s) and searched; Open a chat
window of the Attach tab (SmolLM2-135M with the index
in use, a message answered from the passages). The
oklm serve line of that tab was quoted for a POSIX shell:
on Windows it is quoted for a Windows command line.
Evaluate on a judged set of three documents and two
questions: the set indexed in the window's cache
(%LOCALAPPDATA%\oklm\gui-rag\eval-set-ID.gguf),
3 configurations scored, a row each for dense, sparse and
hybrid. A graph, captions and the reranker were not run there. Without
Allegro's dialog addon (allegro_dialog), the Browse and Add
buttons say so and the paths are typed.
The code
apps/gui/, linked into oklm-gui only
(make gui; every C file of the directory is built, on Linux
and for Windows).
| File | What |
|---|---|
gui.h |
the shell's state (gui_shell), what a tool is
(gui_tool), the list of tools (GUI_TOOLS), the
helpers the tools share |
shell.c |
the entry point: the tool chosen, the command line, the window, the
faces, the event loop and its 30 Hz timer, the drawing,
--screenshot; the program's own path, starting it again,
the bundled documentation, the browser, the clipboard |
http.c |
the HTTP helpers over the client below: one request beside a client
a thread is using (gui_http_call()), whether a server
answers at an address (gui_http_probe()), the first model
of a list (gui_models_first()) |
tool_chat.c |
the chat window, with its model manager, its context slider, its model settings and the restart of its model, its Connect window, its attached files and its commands |
chat_cmd.h |
what the chat window does with its commands without a window, shared
with its test: the place and the features it gives the commands' parser
for what it is linked to, the rows of the list above the message, the
help as a note, the conversation written as Markdown, as JSON and as the
file oklm chat --chat-file reads, the request of
/decide |
chat_remote.h |
what the chat window knows of a server at an address without a window, shared with its test: OKLM's own routes as they are asked once and what an answer says of each, the line that says a server is not OKLM's, such a server's refusal of a prompt longer than its context and of a reply left open, the end of a streamed reply however its JSON is spaced, and what the settings file keeps of a saved server (its name, its model, its context), a key never shown |
chat_listen.h |
what the chat window does with speech without a window, shared with its test: a recording filled by fragments with its level and the pause that ends it, its samples at 16 kHz, the WAV bytes and the request sent to a server and its answer read, a file read as the microphone, who transcribes for each link, the text as it goes in the message |
chat_speak.h |
what the chat window does to read a reply aloud without a window,
shared with its test: what is said of a reply, a text cut into pieces at
the ends of sentences, what /speak was given, who speaks
for each link, the request sent to a server and its refusals, an
answer's bytes as samples, the queue of a playing sink and the sink that
writes a WAV file |
chat_attach.h |
what the chat window does with attached files without a window, shared with its test: a file's kind by its name, base64, a dropped line as a path, a document under its name and cut to a number of tokens, a request's message with its images and sounds, a server's refusal read |
chat_docs.h |
what the chat window does with a document longer than the context, shared with its test: the cache of indexes kept between windows and the folder of the temporary ones, a document's index built from its text or found in the cache, the passages found for a message within a number of tokens and put in order by a reranker, which documents are searched, an encoder that stops when asked |
serve_opts.h |
the options of oklm serve without a window, shared by
the server manager, the chat window and their test: the table of the
options that have a control, the command line made from their values and
read back into them, a profile's object, what applies and why not, the
offload row's two mappings, the memory plan's parameters and its line,
the sampling of a chat request, what a server tells of its settings,
what a restart changes and the question asked before it |
settings.h, settings.c |
the controls of those options, shared by the two tools: the offload row, the sampling block and the Advanced window with its tabs, each control a line of the table, the values kept in step with the widgets at each timer event, what does not apply greyed with its reason |
tool_models.c |
the model manager as a tool: the catalogue's list, the download's thread, the buttons that open another tool on a model |
tool_servers.c |
the server manager: the form, its offload row and its profiles, the thread that asks the servers, the worker that starts, stops, restarts and resizes them and answers the sliders' questions |
tool_convert.c |
the conversion tool, with its file browser |
tool_sessions.c |
the sessions browser: one worker thread for the lists, the search and the reading, a thread a dialog |
tool_launcher.c |
the launcher |
tool_inspect.c |
the model inspector: the JSON of the layout, the planner and the server read into what the window draws, a worker thread for the file, the planner, the server, the run of a prompt and the parts taken out of it, the map and the panels drawn in custom widgets |
inspect_act.h |
its activation views without a window: a token's record made from
the library's or read from /api/activations (its lens and
the range of records a server keeps included), the experts counted over
a run or read from /api/experts, a scan of parts taken out,
a token's text made fit to draw, the colour scale |
tool_train.c |
the training window: the forms (a table of fields), the command built from them, the progress files and the log followed, the curve |
tool_rag.c |
the retrieval manager: its tabs, the thread that runs
rag info and rag search, the build and the
evaluation started through the registry |
rag_cmd.h |
what the retrieval manager does without a window, shared with its
test: the oklm rag command lines, the progress lines of a
build read and counted, a file followed line by line, the evaluation's
table found in a log, the oklm serve line |
A tool is a const gui_tool: its name (what
--tool takes), its title and a line about it (the window's
title, the launcher's row), its window's size, and the callbacks the
shell calls, any of which but build may be left out:
| Callback | When |
|---|---|
usage |
-h, or an argument nobody takes |
option |
for each argument the shell does not know: 1 or 2 arguments taken, 0 for none |
start |
before the window exists: what may fail with a message on the terminal (a model loaded) |
build |
once the window, the faces and the GUI are there: the tool's windows and widgets |
layout |
after build and at each resize |
before_event, after_event |
around the GUI's own handling of each event |
tick |
at each timer event |
ready |
with --screenshot: whether the window shows what is to
be saved |
close |
at the end: threads stopped, state freed; the GUI and the window go after it |
A tool whose drops is set takes what is dropped on its
window: the window is opened with Allegro's drag-and-drop flag (5.2.9
and later; by its value, since the header declares it only under
ALLEGRO_UNSTABLE, which would tie the program to one exact
version of the library), and before_event sees an
ALLEGRO_EVENT_DROP for each file or line of text dropped,
with its text, which the shell frees.
A new tool is one file apps/gui/tool_NAME.c defining
gui_tool_NAME, and one line X(NAME) in
GUI_TOOLS of gui.h: the launcher lists it and
--tool NAME starts it. Its menu entry is one more line, in
packaging/gui-tools.txt (build.md). A tool that starts a server or
a job does it through the registry of background processes
(include/oklm/harness/procreg.h: start detached with a log,
list, follow the log, stop; server.md), so that what it
started is found again by another window or by
oklm serve --list. A registered command line is split back
into its arguments by oklm_procreg_split() (the same
quoting as a shell), which is how the training window finds the progress
file of a run it did not start.
Two things a tool has to know. The system's file dialog is Allegro's
dialog addon (GTK on Linux), linked when it is installed
(HAVE_ALLEGRO_DIALOG): it blocks its caller, so it is shown
from a thread of its own (pick_open() in
tool_rag.c) and the window goes on drawing. GTK sets the
program's locale from the environment: LC_NUMERIC is set to
C before it starts (and LC_ALL, which would
override it, moved to LANG), or a French or German session
would write 0,5 in every JSON number from then on, and
numbers with a decimal point would stop being read
(tool_train.c puts it back too). And no function of the
program may have the name of one a loaded library exports: the string
buffer's hb_free() is linked as oklm_hb_free,
since HarfBuzz, which GTK shapes text with, has an hb_free
of its own.
The client
include/oklm/client/client.h,
src/client/client.c, over nilorea's network module
(ADR-0001: TLS through nilorea).
oklm_client* c;
char err[256], *answer = NULL;
int status;
oklm_client_new("https://host:7430", "KEY", 1, &c, err, sizeof(err));
oklm_client_request(c, "GET", "/v1/models", NULL, &status, &answer);
oklm_client_stream(c, "/v1/chat/completions", body, on_event, user, &status, &answer);
oklm_client_free(&c);- Connections. One connection per request
(
Connection: close), 10-second connect timeout. - Answers. Bodies by
Content-Length, chunked framing, or until the close. - Streams.
oklm_client_stream()passes each server-sent event's data to the callback as it arrives (a JSON chunk, then[DONE]). An answer that is not a stream, an error, comes back whole.oklm_client_cancel()from another thread shuts the stream's connection down, and the stream returns. - Bodies as they come.
oklm_client_get_data()(a download) andoklm_client_post_data()(samples a server sends while it makes them,POST /v1/audio/speech) pass the body of a 2xx answer to a callback piece by piece, a chunk of a chunked answer a call; the body of another status, an error's JSON, comes back whole from the POST. The callback stops the transfer by returning 0. - TLS.
- Certificates are checked against the system's authorities, and the certificate's name against the host (nilorea's handshake).
oklm_client_set_ca()trusts a private authority too.- Verification can be turned off.
- A build without OpenSSL refuses
https://.
Tests
tests/server/test_server.c runs the client against the
server.
- Keys: right and wrong.
- Answers: a whole answer and the same streamed, joined from the events; an error that comes back whole instead of as a stream.
- Cancellation: a long stream cut short from another thread.
- URLs: IPv6, bad schemes, ports and brackets, a refused connection.
- TLS, with a test authority (n_x509):
- the unknown authority is refused;
- accepted once it is given, with a stream over it;
- accepted unverified;
- plain HTTP to the TLS port gets nothing.
tests/rag/test_gui_rag.c runs what the retrieval manager
does without its window: each command line it makes, run as the tool
runs it (a build with its progress file followed, the sources' states
after files changed, an update with the index's own settings that chunks
nothing again, a search in each mode, the evaluation and its table), a
progress line cut in two, and the oklm serve line with its
quoting.
tests/client/test_gui_attach.c runs what the chat window
does with attached files without its window
(chat_attach.h): kinds by extension, base64 against RFC
4648's vectors, dropped lines (file:// addresses with
escapes, another machine's refused), a PDF fixture read and put under
its name, the cut to a number of tokens (never inside a character), a
request's message parsed back (a string without media, else the
image_url and input_audio parts and the text),
a server's figures read for images and sounds, an older server's two
answers to the empty image, the room a context leaves, a thumbnail.
tests/client/test_gui_docs.c runs what the chat window
does with a document longer than the context without its window
(chat_docs.h): which documents of a message are searched, a
text of twenty sections indexed in a temporary file with the hash
encoder, the one fact of a section found for a question about it, within
the tokens allowed and in the document's order, another question finding
another section, a message without a word given the document's start, a
search and a build stopped by their gate, the index's file and its
folder removed, and the block and the line a searched document has in a
message, those of passages spread over the document included.
test_cache: a document indexed in the cache's folder, found
again under another name with nothing embedded and the same passages, a
text changed by a byte and another embedding model each in a file of
their own, a stopped build leaving the kept index as it was, a folder
that cannot be written falling back to a temporary file, and the cache
pruned to its 64 most recent indexes. test_rerank: with the
tiny cross-encoder of the retrieval tests, the passages written are the
first of the reranker's order that hold in the room, and a message about
the whole document is not reranked. An attached agent's room is checked
there too.
tests/client/test_gui_listen.c runs what the chat window
does with speech without its window (chat_listen.h): the
meter's scale, a recording filled by fragments of any size (its level,
the pause that ends it, its end when it is full), its samples brought
from 48 and 44.1 kHz to 16 kHz and compared with the tone they hold, a
device's 16-bit frames mixed to mono, the WAV bytes read back by the
library's decoder, a file read as the microphone, the request received
whole by a socket of the test (its path, its headers, a body full of
zero bytes) and the server's answers read, a server's figures, who
transcribes for each link, and the text as it goes in the message.
tests/client/test_gui_speak.c runs what the chat window
does to read aloud without its window (chat_speak.h): what
is said of a reply, on the sixteen texts the dashboard's page is checked
with (tools/check_dashboard.mjs) and six more, so that both
clients say the same words; a text cut at the ends of sentences, at
blanks, and never inside a character; what /speak was
given; who speaks for each link; a server's figures; the request
received whole by a socket of the test and its answer read chunk by
chunk with oklm_client_post_data() (a sample cut between
two chunks, bytes of zero, a refusal kept whole, a stop); the request
with a first pass cut short, a speed and a voice by its name; the voices
a server offers read in its figures, its own first; an answer's bytes as
samples for every size of piece; the playing sink's queue; the sound of
a reading given as it is made, held while it is paused or wanted whole
and given since, each sample once and in order, its copy for the reply
and the key that tells two readings apart; and the sink that writes a
WAV file, read back by the library's decoder.
tests/client/test_gui_remote.c (88 checks) runs what the
chat window knows of a server at an address without its window
(chat_remote.h): each of OKLM's routes had or not from an
answer (a 404, a 405, no answer, HTML, another route's JSON, a member of
the wrong type), which servers are not OKLM's, the line shown in place
of the figures, the refusals of a prompt longer than the context as
llama.cpp's server and OpenAI word them and the context a refusal tells,
a reply left open refused, the estimate of a conversation nobody
counted, finish_reason read with and without blanks, a
saved server's entry written as the file always had it and with a name,
a model and a context, read back, and a key that is in no row and no
description.
tests/client/test_gui_commands.c runs what the chat
window does with its commands without its window
(chat_cmd.h): the place and the features of each link and
what the parser then says of a line there (a remote server's model
refused, an agent's own commands unknown to the window), the rows of the
list for a name and for an argument being typed, the help as a note, the
conversation written as Markdown and as JSON and parsed back (the notes
left out, the reasoning apart, the file of --chat-file with
each document in its message), and the request of
/decide.
tests/client/test_gui_settings.c runs what the two tools
do with the options of oklm serve without their windows
(serve_opts.h): every line of the table found by its names,
the values each option takes and what is said of one it does not, the
command line in the table's order with a quoted value, the Advanced
window's own line, an option that does not apply left out with its
reason and a tab greyed, a command line read back (other spellings, a
switch, a value refused and left as text, an open quote, a line written
and read back the same), a profile's object, the cache's pair and the
layers' slider, the memory plan's parameters and its line where no model
is read, the sampling of a request, a server's answer read (and an older
server's, which changes nothing; a seed and a length told), a server's
account of its settings compared with the form it was started from, what
a server holds on its device read in its figures and the memory line of
a plan that counts it (it fits; it would not without; it does not fit
even so), what is kept for a model (the options kept, what is written
and read back, an option the command line named, the settings file
rewritten with its other members left alone, a model forgotten), and
what a restart changes with the question asked. 441 checks; the command
lines' quotes are the system's, so the same checks hold under Wine.
tests/harness/test_procreg.c checks
oklm_procreg_split() (a command line written by the
registry read back, a person's quotes, an open quote refused).
make gui-test checks the windows without a person, under
Xvfb, with the tiny model the server test builds: the chat window
(--send TEXT --screenshot FILE.png: one message sent, the
reply and the next figures waited for, the window saved; then the same
with a PDF fixture attached by --attach; then the fixture
video attached on the tiny Qwen2-VL projector of the fixtures, the line
Attached: testsrc.ogv Ogg video, 14 KiB, 2.0 s, 2 frames
and the reply's 183 prompt tokens looked for on the error stream; then
the same video attached after /video fps 2 frames 3, its 3
frames and the reply's 323 prompt tokens; then its sound track
transcribed by the tiny Whisper fixture with
--transcribe-attached; then a recording:
--mic-file with a sound fixture, the tiny Whisper fixture
as --transcribe-model, /listen sent, and the
transcription's line looked for on the error stream; then a text read
aloud: the tiny speech model the server test writes as
--speak-model, /speak with six words sent,
--speak-file in place of the sound, the reading's line
looked for and the file's 48 samples counted), the server manager's form
(--options with the offload row and three tabs of the
Advanced window, --save-profile: the command line, the
Advanced window's line and the profile's options looked
for; the profile loaded by another window, the same command line; a
profile written as the tool kept them before the options had controls,
each option of its text in its control and the rest left as text, with
the memory line), the chat window's model settings
(--settings "--temp 0.3 --top-k 7": the sampling the
request carries; --settings "-ctk f32 -ctv f32" between two
messages: the question, the model restarted, two messages kept and both
answered; a server started with
oklm serve --temp 0.25 --top-k 11: its sampling defaults in
the window's and in its request; the settings kept: a window that sets
--temp 0.3 --top-k 7 -ctk f32 -ctv f32 and
/max-tokens 8 leaves them in its settings file, of mode
600, the next window on the model reads them, serves with that cache and
sends that sampling without being told, and
--settings defaults restarts back to the window's own and
leaves the reply's length alone; a model's option given on the command
line: a window started with
--kv-mem 1 -ctk f32 -ctv f32 --spec-ngram 2 has a cache of
4096 tokens, /model to another model says
automatic again: --kv-mem and makes a cache of four
contexts with the same types and prompt lookup, and /model
back to the first leaves the four contexts, nothing of it in the
settings file; an index's reranker: --rag with
--settings "--rag-rerank ...", the window's server saying
its passages in the search's order before and
reordered by a cross-encoder after), the buttons a
screenshot does not press (below), the
inspector (the line it writes on the error stream; then
oklm run --observe on the model, every line of its file a
record with both blocks' attention; then the inspector with
--prompt and --layer 1, the line of its
activations looked for), the model manager, then the conversion tool
(the tiny model requantized to Q8_0 with --run, the window
saved once the command ended, and the file it wrote there). What the
conversion tool reads of the commands is checked without a window
(tests/harness/test_cli_progress.c: --dry-run
and --progress-json of oklm convert,
quantize and ppl;
tests/io/test_gguf.c: the size of a file before it is
written).
The windows of make gui-test have settings of their own
(XDG_CONFIG_HOME is
build/test_server/gui-config for the whole recipe): what a
person kept for a model is neither read nor replaced by it.
The buttons a screenshot does not press
sh tools/check_gui_form.sh [tiny|canary|device] runs
three flows under Xvfb with the tools' own test aids, no pointer and no
keyboard, with a registry, settings and a state of its own under
build/check-gui-form:
- Open in the server manager...
(
oklm-gui --open-servers): a chat window restarts its model with--temp 0.3 --top-k 7, a cache type and--spec-ngram 2, then presses the Advanced window's link. Checked: the command the server manager is started with (--tool servers -m MODEL --backend cpu -c N --options '...'), its window saved, and its form, kept as the profileopened: the model, the backend, the context, and each option in its control. - Re-keep with form
(
--press rekeep:NAME) with the new options: a server is kept from a terminal (oklm serve --keep), then the form, holding the cache's two types,--kv-mem, four sampling defaults,--spec-ngram,--queueand an option without a control, is given to it. Checked: the question, what the window says after, the unit'sExecStartwith every option, its mode 600, and nothing left onceoklm serve --unkeepran. The service manager is a stand-in (OKLM_SYSTEMCTL) whose units are under the script's directory: nothing is written among the user's units, and the user's own manager is never asked. - Start server (
--press start) with every tab of the Advanced window set: Cache (-ctk,-ctv,--kv-mem,--fixed-context), Sampling (all seven), Vision (--chat-template,--mmproj,--tiles,--ffmpeg), Speech (the transcription and the speech models with their backends, the vocoder), Retrieval (the embedding and reranking models,--rag-k,--rag-mode,--rag-rerank,--rag-dir, with an index in the form), Speed (--draft,--draft-n,--spec-ngram,--prefix-dir), Network (--host,--fixed-model,--queue,--upload-max) and Other (--model-id,--workers). Checked: the server's command line in the registry is the form's, the server's own account of its settings agrees with the form (the backend, the cache's types, the context, not resizable, the seven sampling defaults), its figures name the model, the projector, the two sound models and the index's reranker, the embedding and reranking routes answer, and afteroklm serve --stopnothing answers, the registry is empty and the process is gone. Withcanary, a second start, of a mixture of experts, has the MoE disk tab (--expert-cache,--pin-gb).
tiny (what make gui-test runs: the server
test's tiny models and the fixtures, 18 checks) and canary
(SmolVLM-256M with its projector, Whisper base, OuteTTS 0.2, bge-small,
MiniLM-L6 and Granite 3.1 1B A400M, 22 checks) both passed on
2026-11-15, on the CPU.
The options a device alone takes have a mode of their own,
device (make gui-device-test, not part of
make gui-test, which passes without a GPU): on each device
backend that answers, one server at a time is started from the form with
Start server, compared with what it tells and stopped
again.
| Start | The form | What the server tells |
|---|---|---|
| Qwen3-0.6B, some layers | --gpu-layers 20 -ctk q8_0 -ctv q8_0 |
GET /api/context:
"n_gpu_layers":20,"n_layer":28,"n_devices":1;
device_memory of /api/stats on that
backend |
| Granite 3.1 1B A400M, experts on the CPU and a device tier | --n-cpu-moe 8 --expert-gpu 0.25 |
"n_cpu_moe":8 and expert_gpu, the tier's
bytes, at most the 0.25 GiB asked; expert_gpu.slots of
/api/stats |
| Qwen3-0.6B over two devices | --devices 2 |
"n_devices":2; where one device answers the start is
skipped with a line |
The mode ends with a full context in a chat window on each backend:
Qwen3-0.6B, its 28 layers on the device, at 256 tokens
(room: a context of 512 tokens in place of 256,
the context: 512 tokens asked, 512 now,
continued: piece 2 of the reply, two pieces written), then
at 1536 tokens with --kv-mem 180, a cache of 1632 tokens
that holds no larger context, on a conversation of six exchanges
(compacting: 12 of 13 messages become a note, by itself,
the reply gone on with, the next message answered).
Each start also checks that the server's command line in the registry
is the form's, the window's line
(the server agrees with the form: --backend --gpu-layers -ctk -ctv -c --fixed-context,
with --n-cpu-moe, --expert-gpu or
--devices in it) and that nothing is left after
oklm serve --stop. Two devices are Vulkan's with
OKLM_VK_DEVICES=all (a discrete and an integrated card),
else two OpenCL devices. On 2026-11-15, on OpenCL and Vulkan with an RX
7900 XT and an Intel UHD 750, the five starts passed. Without a device
backend the mode says skipped: no device backend answers
and ends with status 0.
With Qwen3-8B Q4_K_M in the store the mode goes on to the memory line, on each backend:
| Run | What is checked |
|---|---|
| the form at 40960 tokens, Start server | the line:
a cache of 2.4 contexts ... It fits with a smaller cache: 2.4 contexts where four are kept when there is room.;
the window writes
the cache: 97664 tokens planned by the memory line, 97664 made by the server (0.0 % apart),
the server's cache_tokens being what the line planned |
| the form at 131072 tokens, Start server | The device holds the model with less than one context of cache beside it: 28 of its 36 layers go there with 14.3 GiB of the cache, the other 8 keep 4.1 GiB of it in memory, slower than the whole model on the device. It fits with 28 layers on the device, the others on the CPU, and a smaller cache: 1.0 contexts ...;
the server tells "n_gpu_layers":28, its cache is the 133664
tokens the line planned, and it answers |
the form at 131072 tokens with --gpu-layers 36 |
It does not fit: what is left for the cache holds 0.7 contexts, less than one; a context of 131072 tokens takes 18.0 GiB. |
| the form at 16384 tokens, Cache size 4096 | a cache of 4.0 GiB as asked, 1.8 contexts; the server's
cache is the line's, to the token |
the chat window at 4096 tokens, then
-c 40960 --gpu-layers 36 applied, then
-c 131072 |
the line counts what the model served holds and says a smaller
cache; the model is restarted
(restarted in 1.5 s; 2 messages kept), its server's cache
the 97664 tokens planned; 131072 tokens with the 36 layers still asked
for does not fit, and Apply is refused
(these settings do not fit, as the memory line says) |
a chat window on Qwen3-0.6B, /model to Qwen3-8B, for
which -c 131072 is kept |
served in 2.7 s with the settings kept for it; now -c 131072,
the window's server: vulkan, 28 of 36 layers on a device,
its cache 133664 tokens, the conversation going on |
the same, with -c 131072 --gpu-layers 36 kept for
Qwen3-8B |
served ... with the window's settings, those kept for it left aside: with a context of 131072 tokens it does not fit: what is left for the cache holds 0.7 contexts, less than one; ...,
the conversation going on |
On 2026-11-15 these passed on both backends (68 checks in all, the starts above included).
The mode ends with what shares the device (The memory line), a server started from the
form on each backend: SmolVLM-256M with its projector, and Qwen3-8B with
Qwen3-0.6B as its draft model, both at 8192 tokens. Checked for each:
the line says Counted beside the model: ... on the device,
and the server's cache_tokens is the cache the line
planned, within 2 % (the window writes both:
the cache: 806992 tokens planned by the memory line, 806992 made by the server (0.0 % apart));
for the projector, that the server's figures say it is on the device
("vision_device": true) and that its share as the line
counted it is the one the server tells in device_memory
(beside the model on the device: 619033600 bytes counted by the memory line, of which the draft model 0; 619033600 told by the server).
Then the models beside the model where their backends put them, a server of Qwen3-0.6B at 8192 tokens started from the form on each backend, with bge-small-en-v1.5 Q8_0 and MiniLM-L6 F16 from the store:
| Start | What is checked |
|---|---|
--embed-model ... --embed-backend B, the embedding
model on the device |
the line says
Counted beside the model: 94 MiB on the device, 26 MiB of memory;
the server's figures have "embed":98637824 in
device_memory, the share the line counted
(beside the model on the device: 98637824 bytes counted by the memory line ...; 98637824 told by the server);
its cache is the one planned, within 2 % |
--embed-model ... --rerank-model ..., both on the
CPU |
the line says
Counted beside the model: 232 MiB of memory and nothing on
the device; the cache is the one planned; what the line counted for the
two contexts is what the server holds for them, to the byte
(beside the model in memory: 162264064 bytes counted by the memory line for what the models on the CPU work with; 162264064 held by the server's embedding, reranking and transcription models now) |
Then a model loaded in place of another with nothing kept for it, in the chat window, on each backend:
| Run | What is checked |
|---|---|
a window on Qwen3-8B, --gpu-layers 20 applied, then
/model to Qwen3-0.6B |
the window's server: vulkan, 20 of 36 layers on a device,
then
served in 0.5 s with the window's settings; automatic again: --gpu-layers; now -c 4096
and 28 of 28 layers on a device: the count set for 36
layers is not Qwen3-0.6B's; the conversation goes on |
a window on Qwen3-0.6B at 32768 tokens, the device held to 8 GiB
(OKLM_GPU_MEM_MB=8192), then /model to
Qwen3-8B |
the device holds Qwen3-8B with 0.6 contexts beside it: it is loaded
with fewer layers there
(the window's server: vulkan, 27 of 36 layers on a device, ... a context of 32768 tokens,
cache: 34000 tokens, 1.0 contexts) and answers the next
message |
Then a model larger than its device, the device held
to 4 GiB (OKLM_GPU_MEM_MB=4096) and Qwen3-8B Q4_K_M started
from the form at 4096 tokens on each backend: the line says
The model is larger than the device: 22 of its 36 layers go there with 407 MiB of the cache, the other 14 keep 259 MiB of it in memory. It fits with 22 layers on the device, the others on the CPU, and a smaller cache: 1.2 contexts where four are kept when there is room.,
the server tells "n_gpu_layers":22 and its cache is the
line's to the token
(the cache: 4736 tokens planned by the memory line, 4736 made by the server (0.0 % apart)).
Then what the line counts beside the model, each against the server's own figure:
| Start | What is checked |
|---|---|
SmolVLM-256M with --mmproj, on the CPU backend |
the line says
Counted beside the model: 284 MiB of memory; what it
counted beyond the file is what the server holds for the encoder
(beside the model in memory: 108003328 bytes counted by the memory line ...; 108003328 held by the server,
its host_memory.vision) |
Qwen3-0.6B at 8192 tokens with --speak-model OuteTTS
0.2, --speak-vocoder, --preload, the speech
model on the CPU, on each backend |
Counted beside the model: 1.3 GiB of memory; the
757346304 bytes counted beyond the files are the server's
host_memory.speak and speak_call together; the
device's cache is the one planned, within 2 % |
Qwen3-0.6B on vulkan with --embed-model
bge-small --embed-backend vulkan1, where two Vulkan devices
answer (OKLM_VK_DEVICES=all: an RX 7900 XT and an Intel UHD
750) |
the line says
On vulkan1: 94 MiB of 84.1 GiB, for the embedding model.;
the server's side_devices has
"backend":"vulkan1" with "embed":98637824, the
bytes the line counted there
(beside the model on other devices: 98637824 bytes counted by the memory line; 98637824 told by the server);
the model's cache is the one planned |
a chat window on Qwen3-0.6B, --model-settings,
bge-small and MiniLM-L6 installed (OKLM_MODELS_DIR) and no
option naming them |
its line ends
Counted beside the model: ... of memory. Counted: the models a long document is searched with, loaded when one is attached.;
with --embed-model hash --no-rerank it says nothing of
them |
Last, one server, three lines: the form's memory
line, the line under the selected server's Context
slider in the server manager (--press context:N) and the
line under the slider of a chat window attached to that server
(--url) are checked against each other and against the
server's own cache_tokens (GET /api/context),
on each backend. The two tools write the line they show
(oklm-gui: the context line: 40960 tokens, cache 97664 tokens, grey: ...),
and the figure there is the cache the server told for that line.
| Server | The form | The server manager's line | The chat window's line |
|---|---|---|---|
| Qwen3-0.6B at 8192 tokens, started from the form | fits |
8192 tokens: grey, cache 161216 tokens, the server's | the same |
| Qwen3-8B at 40960 tokens, started from the form | fits with a smaller cache |
40960 tokens: grey, cache 97664 tokens, the server's; 131072 tokens:
red, It stops near 97664 tokens, where the cache ends. (the
server that runs keeps its 36 layers on the device) |
the same two lines |
the same model at 131072 tokens with --gpu-layers 36,
in the form |
red, does not fit |
||
| the same model at 131072 tokens, the layers left to what fits, in the form once that server is stopped | grey:
The device holds the model with less than one context of cache beside it: 28 of its 36 layers go there ...,
what a server started then would place (while the first server holds the
device the form finds no layer that leaves it a context, and says the
model would run on the CPU) |
||
Qwen3-0.6B at 16384 tokens with --kv-mem 64, started
from a terminal |
16384 tokens, the context it runs with: red,
It stops near 576 tokens, where the cache ends. |
the same |
The last row is where the two tools parted before: the server manager
said nothing of the size a server has, and the chat window judged it.
With tiny, on the CPU, a server whose cache of 1 MiB holds
8192 tokens at a context of 16384 has the three lines red too: a cache
in host memory that ends before the context was said by neither
tool.
On 2026-11-16 the whole mode passed on OpenCL and on Vulkan (RX 7900
XT): 174 checks (110 before these and the models beside the model);
tiny, 22 checks.
The same under Wine. On 2026-11-15 the Windows build
(make MINGW=1 BUILD=build/mingw tests apps gui) was run
under Wine 11 and Xvfb in a prefix of its own, with these unattended
options and no pointer, on the tiny models: the settings kept (written
in %APPDATA%\oklm\gui-servers.json, read by the next window
with its server's cache and its request's sampling,
Defaults), the index's reranker reaching the window's
server, Open in the server manager... (the command, its
window saved, its form kept as a profile; the options' line is quoted as
Windows quotes it), Start server with the Cache,
Sampling, Vision, Speech (the transcription model), Retrieval (the
embedding and reranking models), Speed, Network and Other tabs set
(the server agrees with the form: --backend -ctk -ctv -c --fixed-context --temp --top-k --top-p --min-p --repeat-penalty --seed -n,
then stopped by oklm.exe serve --stop), and Re-keep
with form on a server kept by
oklm.exe serve --keep (the Run key's value
then holding the form's options; removed by --unkeep): 17
checks. Not run there: the canary models and the speech model's options
in a start, an index in the form, the restart on a device from the model
settings, and tools/check_gui_kept.sh, which needs systemd.
The pictures' script is the Linux build's.
A start with nothing named
sh tools/check_gui_start.sh, which
make gui-test runs, starts the chat window with nothing
named, under Xvfb on the CPU, with settings and a models' folder of its
own (build/check-gui-start), beside a server of the tiny
model it starts on a port the windows search, a second one that asks for
a key, an agent that listens, and with SmolLM2-135M linked in that
folder where it is under models/:
| Case | What is checked |
|---|---|
| nothing named | while the window is open no port more answers on loopback (its own
server would take one) and its error stream names no model; saved, it
says
started with nothing named: no model is loaded; N servers on this machine, M installed models |
| a message before a choice | --send "hello" is not sent:
oklm-gui: --send: Not sent: nothing is loaded yet. ...,
nothing is answered, and the window is saved, not left waiting |
| a server chosen | --press choose:PORT --press use:
chosen: the server at http://127.0.0.1:PORT: nothing is loaded by the window,
and the message sent after is answered; a port nothing answers on
selects nothing, and use then presses nothing |
| a model chosen | --send "hello" --press choose:smollm2-135m --press use --send ...:
chosen: the model smollm2-135m: judged to fit, loaded in this window,
then
the window's first model: SmolLM2-135M-Instruct-Q8_0; the conversation stays, and the 5 bytes of the message being written
(the message refused before the choice is still in its field), no
starts again, and the message sent after is answered;
skipped with a line without the model |
| a file chosen | --press choose:file:build/test_server/tiny.gguf --press use:
chosen: the file ...: listed, not loaded yet, then loaded
in the window and answering; with
--press choose:mmproj:.../tiny-mmproj.gguf before
use,
the window's first model: tiny, with the projector tiny-mmproj.gguf;
choose:file selects the row of the dialog,
choose:mmproj nothing before a file is chosen |
| a load refused | the same file in a window started with
--kv-mem 1 -c 16384:
the model: ... asked, not loaded: with a context of 16384 tokens it does not fit: that cache holds 0.5 contexts, less than one; a context of 16384 tokens takes 2 MiB.,
nothing served, and the message after it is
Not sent: nothing is loaded yet |
| a saved server | a server started with --api-keys, kept in the check's
gui-servers.json under http://localhost:PORT
with its key:
--press choose:http://localhost:PORT --press use links the
window and the message is answered; with another key kept,
chosen: the server at ... asks for a key: handed to the Connect window,
as for its row among the servers of this machine
(choose:PORT) |
| an agent | oklm agent --listen on a port outside those searched
for servers: --press choose:PORT --press use gives
chosen: the agent at http://127.0.0.1:PORT: nothing is loaded by the window |
| nothing to choose | an empty models' folder and no server answering: the window says
0 servers on this machine, 0 installed models and opens on
its Models window; skipped with a line when another server answers on
the machine |
The window on a server that is not OKLM's
sh tools/check_gui_remote.sh, which
make gui-test runs, links the chat window to a server that
is not OKLM's, under Xvfb, with settings of its own
(build/check-gui-remote): 115 checks on a stand-in
(tools/standin_openai.py: GET /v1/models and
POST /v1/chat/completions, a key asked for, every other
route a 404, each request written down without its headers) and on
oklm serve, and 15 more on llama.cpp's server where it is
built (build/llama.cpp/build-server/bin/llama-server, or
LLAMA_SERVER) and Qwen3-0.6B is under models/,
on the CPU and on loopback; that part is skipped with a line
otherwise.
| Case | What is checked |
|---|---|
| the routes | oklm-gui: the server at URL is not an OKLM server (no /api/stats /api/context /api/commands /api/model /api/compact): the context is the window's setting, 32768 tokens; the model asked is standin-a;
two messages answered, their requests naming the model; in the
stand-in's log each of the five routes once and no sixth request under
/api/; nothing cannot be reached |
| the context | /context 8192, then
--press context:16384 --press apply-context:
the context: the window's setting, 16384 tokens, the line
under the slider grey with where the window compacts; after a reply of
400 tokens, the slider at 256: red,
The conversation holds 404 tokens: with 256 it will be cut at 256 tokens.;
no request under /api/ for any of it |
| a reply gone on with | a stand-in that refuses a request whose last message is the
assistant's: --press continue twice gives
continued: the server takes no reply left open (status 400): the model is asked to go on in a message
once, one request that left the reply open, two that asked in a message,
and the reply saved by /save is w1 ... w20 in
order |
| a compaction | /compact on a stand-in that refuses unknown members:
compaction: the server does not count texts: an estimate,
the summary asked with chat_template_kwargs once, refused,
then without it; compacted: N messages into a note, and the
message after it answered |
| a full context | a stand-in of 260 tokens: the seventh message is refused
(request (278 tokens) exceeds the available context size (260 tokens)),
the context: the window's setting, 260 tokens, as the server told it,
compacting: 12 of 13 messages become a note, by itself, and
the message is answered |
| the model | a stand-in that lists two:
the server lists 2 models: chosen in the side panel, or by /model NAME;
--press model:standin-b and the next request names it;
/model standin-c, which it does not list, is refused and
/model standin-a taken; the chooser has no row for
standin-c; in a second window
--shot-when "commands:/model standin-" gives
the list above the message: 2 rows, the first /model standin-a |
| the models of a server not connected to | in a window started with nothing named,
--press fill:url=... fill:key=... --press list-models --press pick-model:standin-c --press save:
models listed: 3, the first standin-a, and the entry saved
has "model":"standin-c"; with a key the server refuses,
models not listed: URL does not list its models: the server refuses the key (status 401: Invalid API Key). The name can be typed.,
the key typed in no output and nothing offered; on a port nothing
listens on,
No server answered at URL: its models are not listed. The name can be typed. |
| a saved server | --press fill:url=... fill:key=... fill:name=Stand-in fill:model=standin-b fill:ctx=8192 --press save
in a window started with nothing named: the settings file has mode 600,
which the window reads back
(the settings file is its owner's alone), and the entry
with its key, name, model and context; the next window lists it
(choose:URL), use connects with the key kept
(... 8192 tokens, kept with it; the model asked is standin-b),
no request refused, /context 4096 and the model chosen in
the side panel written back in the entry; forget:URL leaves
neither the key nor the address in the file, and the row is gone |
| no count of the tokens | a stand-in whose streams end without usage: six
requests ask for it, each reply gives
answered: done in ... s, no count from the server: about N tokens by estimate,
said once: this server tells no count, the conversation is counted by estimate
once, and
compacting: N of M messages become a note, by itself. One
that answers 400 to stream_options:
the server refuses stream_options (status 400): asked again without, and no longer sent it
once, one request with the member and two without, both messages
answered. On oklm serve the reply's line still ends with
the prompt N tokens and nothing is by estimate |
| a sound | --attach tests/fixtures/mm/tone.ogg and a message: the
stand-in decodes the part of the request,
audio=wav:pcm1/1ch/16000Hz/16bit, and answers. A stand-in
that refuses a sound:
not answered: error 400: audio input is not supported by this model. The images, sounds and videos are left out of the next messages.,
one request with the sound, and the next message answered without
it |
| the sampling | a stand-in that refuses top_k and seed by
name, the window sending /top-k 20,
/top-p 0.9, /seed 7:
the server refuses top_k (status 400): asked again without, and no longer sent it
once, then the same for seed; the first message takes three
requests, the second one, which carries top_p alone; both
are answered. A stand-in that answers every request 400 without naming a
member:
not answered: error 400: the request is not one this server answers,
one request, nothing left out |
| a server that is late | a stand-in that closes its first connection without an answer:
the server at URL did not answer: what it is is found out when it does,
one connection dropped and no other route tried on it; then
the server at URL answers now: not an OKLM server, the line
that says what it is, the four other routes asked once, both messages
answered and the last naming the model. A port nothing listens on: the
same first line,
not answered: the server cannot be reached, and the window
is taken for no kind of server |
oklm serve |
the same window on OKLM's server: no such line, its sampling
defaults read, its context resized by the server,
compaction: the server's tokenizer counted 2 messages |
| the key | in none of the windows' error streams and outputs (the stand-in's and llama.cpp's part each) |
| llama.cpp's server | started with --api-key: the line;
Count from one to forty in words. cut at 24 tokens and gone
on with by Continue in a request that leaves the reply
open (no refusal), the saved reply one text across the two pieces;
/context 4096; a compaction by estimate and the message
after its note answered; saved with its key (mode 600) and chosen at a
start with nothing named, 4096 tokens, kept with it,
answered |
The Advanced window in a small window
sh tools/check_gui_advanced.sh, which
make gui-test runs, opens each tab of the Advanced window
in a window of 760 by 480 pixels, under Xvfb on the CPU with settings of
its own (build/check-gui-advanced): the nine tabs of the
server manager
(--tool servers --size 760x480 --options '...' --advanced TAB,
every option given a value) and the eight of the chat window
(--size 760x480 --press advanced:TAB; it has no
Network tab, which
not pressed: advanced:network says). Each window is saved
at 760 by 480, and the seventeen pictures were read: no control is cut
and none lies over another, the densest being Retrieval
in the chat window (eight rows, the two lines of what is greyed, the two
buttons). Then the chat window's own:
| Case | What is checked |
|---|---|
| the Other field | --settings "--workers 3 --deterministic -ctk f32": the
question names -ctk, the other options, the model restarts
(now -c 512 -ctk f32 --workers 3 --deterministic), both
messages are answered, and
kept for tiny.gguf: {"-ctk":"f32","-n":"8","other":"--workers 3 --deterministic"} |
| read again, refused, forgotten | the next window says
kept for this model, read: -ctk -n, the other options;
--insecure, --workers many and
--expert-loaders 2 without an expert cache are each refused
with the reason; --settings defaults ends with
kept for tiny.gguf: nothing |
| the MoE disk options | a window started on the tiny mixture of experts with
--expert-cache 0.001 --pin-gb 0 answers, and
--settings "--expert-cache auto --pin-gb 0.0001 --expert-loaders 0 --expert-predict off"
restarts it with them
(now -c 512 --expert-cache auto --pin-gb 0.0001 --expert-loaders 0 --expert-predict off),
the conversation going on |
A full context with the voice and with pictures
sh tools/check_gui_full.sh, which
make gui-test runs, has replies cut and gone on with while
something else goes on, under Xvfb on the CPU with the canary models
that are under models/ (a case whose model is missing is
skipped with a line, the whole without SmolLM2-135M). The voice is the
tiny speech model of the server test: its sound is not listened to; a
saved window writes each sentence it gives the voice
(oklm-gui: to the voice: ...), and
tools/gui_said_words.py compares their words, in order,
with the words of the reply /save wrote.
| Case | What is checked |
|---|---|
| a larger context | SmolLM2-135M at 256 tokens, temperature 0, Read aloud on:
room: a context of 512 tokens in place of 256,
continued: piece 2 of the reply, and the 383 words of the
reply given to the voice once, in order; the first sound is told once,
at a time counted from the message |
| a compaction | a conversation of six exchanges
(tests/fixtures/chat/six-exchanges.json) in a context of
1536 tokens whose cache holds no larger one (--kv-mem 36):
compacting: 12 of 13 messages become a note, by itself, the
reply goes on, its 1036 words are given to the voice once, and no
sentence of the note is |
| Continue pressed | --auto-continue off, then
--press continue: the reply says
The reply was cut by the context: /continue goes on with it.,
the piece asked for is read from where the voice stopped, the 383 words
once |
| a spoken conversation | Whisper base hears samples/jfk.wav
(--talk --mic-file), the reply is cut at 192 tokens and
gone on with, sentences of the second piece are said, and the window
ends on listening |
| pictures | SmolVLM-256M with its projector at 1536 tokens: the six exchanges,
then two pictures with a question, a second question,
/compact:
compacted: 8 messages into a note of N bytes, 8 kept (the
message with the pictures and its reply among those kept), and the
message after it is answered from a prompt that is shorter (1206 tokens
after 1486) and still holds the pictures |
It passed on 2026-11-16, 19 checks. On a device the same is part of
make gui-device-test (above): Qwen3-0.6B
with its 28 layers on the device, a reply cut at 256 tokens gone on with
after the context was made 512 in place, and, with a cache held to one
context (--kv-mem 180 at 1536 tokens), room made by a
compaction, the reply going on and the next message answered.
The presses of the other tools
The launcher, the training window, the sessions browser and the
retrieval manager take --press as the server manager and
the chat window do: a button pressed without a person, as its handler
does it, each press in its turn. One piece of the shell carries them
(gui_presses, apps/gui/gui.h): a press waits
for what it presses (a button that is not enabled yet, a list that is
not read, a run that has not ended), two minutes at most, then the next
one comes; the window writes oklm-gui: pressed: WHAT on the
error stream, with what it said after (oklm-gui: said: ...)
and the question it asked (oklm-gui: asked: ...), or
oklm-gui: not pressed: WHAT: what it presses did not come,
or nothing of that name can be pressed; with
--screenshot the window is saved after the last press. A
press whose word the tool does not know is refused as an option, with
the usage. The presses are in each tool's options: the launcher (open:TOOL,
docs), the training
window (stop, merge, export,
export:FILE, chat), the sessions browser
(select:ID, select:live, attach,
resume, export, export:FILE,
delete, yes, no), the retrieval manager
(sources:PATH, build, eval:DIR),
and, for the chat window, export, export:json
and attached:N beside the others (Options).
sh tools/check_gui_press.sh, which
make gui-test runs, drives them on Linux under Xvfb, on the
CPU with the tiny model, in a registry and settings of its own
(build/check-gui-press), and ends what the presses
started:
| Tool | The presses | What is checked |
|---|---|---|
| the launcher | docs, open:models |
the documentation's index handed to a stand-in for
xdg-open (or the window saying there is none), the model
manager running after the launcher ended; open:nothing
refused with the usage, status 2 |
| the training window | a run from scratch started by --start in a window that
is then ended; stop in another |
the second window finds the run, Stop ends it
(stopped on request at step N of 200000: a checkpoint is written),
nothing of it runs after |
a LoRA run of 20 steps by --start, then
merge, export:FILE, chat |
each press waits for its button: the merged model written, the adapter copied to the file (the same bytes), the chat window running on the merged model | |
| the sessions browser | an agent listening on a port: select:live,
attach; select:nothing |
the chat window started on the agent's port and running; a session
that is not listed is not selected
(not pressed: select:nothing: nothing of that name can be pressed) |
select:ID, export:FILE,
delete, no; then select:ID,
delete, yes |
the Markdown file holds the session; Keep it leaves
the session; the question (Remove this session for good?),
then Delete it removes its transcript and not the
other's |
|
| the retrieval manager | --model hash, sources:DIR,
build; then eval:DIR |
the index built and answering oklm rag search; the
judged set indexed in the window's cache and scored |
On 2026-11-16 it passed, 34 checks.
tools/check_gui_wine.sh runs the Windows build's tools
under Wine and Xvfb and checks what they leave (files, processes,
answers on a port, what a window wrote on its error stream). Every
button is pressed by these aids, or by --send,
--start and --query: the script holds no place
of a window, so a control that moves does not break it. What is not the
tools' own is done as a person without a pointer does it: the system's
file dialog is answered from the keyboard (the name typed, Enter), a
window that stays is closed with Alt+F4, and the two files given to the
chat window are a drop of the display's, which Wine hands on as the
system's. The chat window (three unattended runs, the files dropped, the
message that waits for them with attached:2, the two
exports through the dialog with export and
export:json); the chat window started with nothing named
(group start: beside a server of the Windows build, nothing
loaded and the server listed, a message before a choice not sent, the
server's row chosen with choose:PORT and answering, a GGUF
file given by choose:file:PATH judged, loaded in the window
and answering with the message written before it kept, a load the memory
line refuses, and the row of the system's file dialog pressed, the
dialog answered from the keyboard with the file's name and the file
loaded); the launcher from a staged folder (open:sessions,
docs: a tool that outlives it, the documentation's page);
the model manager's list; the server manager (a server started from the
form, restarted, its Dashboard and Chat, stopped; another found by its
port and stopped); the conversion and the inspector's unattended runs;
the training window (a run started, left, found again by a window whose
stop ends it; the Fine-tune, LoRA and RL tabs, then
merge, export through the dialog and
chat); the sessions browser (select:live and
attach on a listening agent, select:ID and
resume in a console, export through the
dialog, delete answered no, then
yes); the retrieval manager (sources:DIR and
build, a search by --query,
eval:DIR); a server kept across logins; a picture sent to a
vision model. On 2026-11-16, under Wine 11 in a prefix of its own
(WINE_GUI_PREFIX=build/mingw/wine-prefix), after
make MINGW=1 BUILD=build/mingw tests apps gui: 58 checks
passed in 186 seconds (the sound group skipped: no
ultravox-1b in the canary set); the group
start, added after, was run alone the same day
(CHECKS=start): 13 checks in 34 seconds. Its screenshots
stay in build/mingw/wine-gui-check/shots (testing.md). They are Wine's runs:
nothing here was run on Windows itself.
By hand. Each tool was driven under Xvfb with xdotool, on the CPU, and its screenshots read, on 2026-11-12 and 2026-11-13:
- the launcher: Documentation on the bundled
index.html; a tool started and left running when the launcher ended; the same from the AppImage (the documentation's 97 files copied and opened from the copy, the tool in a mount of its own); the Windows build's launcher under Wine (every tool opened, the documentation's page handed toxdg-open); - the chat window's files: a picture attached by
--attach, by its path typed and Enter, by the path and Attach, and by a drop from a GTK window (tools/gui_drag_files.py, two files at once), described by SmolVLM-256M with the catalogue's projector, with--mmproj, and through a server started with--mmproj; a WAV written down by Ultravox; a text and a PDF answered from by SmolLM2-135M, on its own server and on another; each refusal read (no projector, a server without one, a sound to a vision projector, a binary file, a file that is not there); a document of 26000 tokens in a context of 4096 cancelled, then its beginning sent; the window at 800x500 with five files on three rows; - the chat window's videos, on 2026-11-14, unattended
(
--attach,--send,--screenshot, the CPU): the fixture video on the tiny Qwen2-VL projector (above), on the tiny Ultravox projector of the server test (its sound track: 33 prompt tokens, answered) and on the tiny model without a projector (refused before sending, the reason above); Qwen2-VL 2B Q4_K_M askedWhat does the video show at its start and at its end?about a clip of four frames (ROADMAP_DONE.md): 418 prompt tokens and the 64 tokensoklm describeandoklm servegive for it; the same clip as MP4 refused without--ffmpeg, attached with it (MP4, 72 KiB, 4.0 s, 4 frames), and attached without a thumbnail to a server at an address started withoklm serve --ffmpeg, which answered. The file dialog, a drop and the path's field were not driven with a video: they hand a path to the code--attachruns; - the chat window's
/videoand a video's Transcribe, on 2026-11-14, unattended (the CPU, the tiny model with the tiny Qwen2-VL projector, the fixture video, which has a 440 Hz tone for sound):/videoalone,/video fps 0refused,/video default;/video fps 2 frames 3then/attach: 3 frames in the line, 323 prompt tokens where the server's own sampling gives 2 frames and 183; with the tiny Whisper fixture and--transcribe-attached, the video's track transcribed by the window's server (transcribed 2.0 s in 0.1 s by the window's server, with tiny-whisper), the text in the message, the video still in the row with its Transcribe button (the picture was looked at). The button's press itself was not driven (the option calls what it calls), nor the window's own recognizer on a video (the server's was), nor a video without sound; - the chat window:
make gui-test's run; messages sent with Ctrl+Enter and Shift+Enter; a server found and connected to; a listening agent attached, the Write tool's question answered with Allow; a session read; back to the window's own model; a model downloaded (99 MiB/s), used and another deleted from the Models window; - the model manager: a model downloaded and deleted, the licence box;
- the server manager: the form, a profile saved, loaded and deleted (mode 600), a server started, resized, restarted and stopped, its log followed, its dashboard and its chat window;
- the offload row, the Advanced window and the model settings, on
2026-11-15. Unattended (
--options,--advanced,--model-settings,--settings,--screenshot, the pictures read): the form on SmolLM2-135M and on Granite 3.1 1B A400M on the CPU (Layers and MoE CPU greyed, the memory line withits keys Q8_0 and its values Q4_0), on Qwen3-8B with the device's plan (20 of 36 layers, the pictures above); the chat window's panel on a server started with--temp 0.25 --top-k 11 -ctk q8_0 -ctv q8_0; settings that cannot hold (--chat-templatenaming a file that is not there:served as before, the next message answered); and the restart on a device, Qwen3-0.6B on OpenCL (--gpu-layers 20 -ctk q8_0 -ctv q8_0between two messages: restarted in 1.0 s, 20 of 28 layers in the panel, both messages answered) and on Vulkan (-ctk q8_0 -ctv q8_0: 0.5 s). By hand with xdotool, on the CPU: the model's name pressed, Cache set to 8-bit, Apply, Cancel (Nothing was restarted), Apply again with the box checked and Restart, then 4-bit and Apply, which restarted without the question; Restart pressed while a reply of 1024 tokens was being written (The reply being written is waited for, half a minute at most; then the model restarts, the restart once it ended, the reply still there; the half minute itself was not waited out); Advanced..., the values' type set toq8_0beside keys inq4_0(the panel's chooser then readsother (Advanced), the memory lineits keys Q4_0 and its values Q8_0), the Speech tab with its greyed fields and their reasons, Close;7typed in Top-k (the side panel gainsSampling: top-k 7); the panel's Context moved to 32768 (the line red, larger than the 8192 tokens the model was trained with); in the server manager on Granite, Layers clicked to20/24with the device's plan,alltyped in MoE CPU, the MoE disk tab with--pin-gb, --expert-pack: no expert cache. Open in the server manager..., Re-keep with form with the new options and a server started from the form with them are pressed bytools/check_gui_form.sh(above); - the memory line on a device, on 2026-11-15, on the RX 7900 XT
(unattended:
--model-settings,--settings 'ask ...', the line read on the error stream). Qwen3-0.6B served on Vulkan, then on OpenCL, its panel set to an 8-bit cache:On the device: 1.9 GiB of 18.3 GiB, 28 of 28 layers there. Counted as free there: 1.5 GiB free now and the 18.2 GiB the model served now holds, which a restart frees first. It fits.(1.6 and 18.1 GiB on OpenCL). Qwen3-8B on each, its panel at 32768 tokens with an 8-bit cache:14.3 GiB of 18.3 GiB ... It fits, where the device's 1.5 GiB free alone would have refused it; at 40960 tokens with all 36 layers asked:27.3 GiB of 18.3 GiB, 36 of 36 layers there, as asked. Counted as free there ... It does not fit, on both; at 262144 tokens: the host's cache of 144 GiB against 116 GiB available,It does not fit. What the server says it holds against the system's own count of the card's memory while the window served: 18.2 GiB told, 18.0 to 18.4 GiB more in use than before (Qwen3-8B, the two backends). A real Apply on Vulkan (Qwen3-0.6B,--gpu-layers 20 -ctk q8_0 -ctv q8_0): restarted in 0.6 s, 20 of 28 layers, the server then holding 6.1 GiB beside 13.7 GiB free; then-c 1048576 --gpu-layers 28, which the line refuses: nothing restarted, the next message answered as before; - the conversion tool: a GGUF file requantized and a checkpoint converted in two steps, both measured; a conversion to IQ2_XXS stopped half-way (the process gone, no file left); a text shorter than one window (the measure fails and the log says why); the browser; the window at 900x600 and 1500x900;
- the inspector: a dense model and a mixture of experts as files, a
block pointed at and clicked, the context dragged to the trained one,
--ram 1500(red, "does not fit"), a device backend (the planner's query only), a running server while it answered; the window at 1400x900 and 800x520; - its activation views, on the CPU with SmolLM2-135M (dense) and
Granite 3.1 1B A400M (a mixture of experts): a prompt typed and run with
Enter, the tokens stepped with Left and Right, a block clicked, a cell
of its attention and an expert pointed at (the figure read out); the
same models served with
--observe, the window reading them while a request of 400 tokens decoded and a block clicked meanwhile; a server without--observe(the note, the tensor table kept); - the training window: a run from scratch; LoRA of rank 4 on SmolLM2-135M (the merged model written, a chat window on it, the export dialog); a run of 400 000 steps left by a closed window, found again and stopped; RL on the model trained from scratch; the Advanced tab's three stages;
- the sessions browser: sessions listed and sorted, read folded and in
full, searched and walked, both exports through a stand-in
zenity, Resume with a stand-in$TERMINAL, Copy, Delete kept then confirmed, a live agent found within two seconds and attached to; withoutzenity,kdialogor a terminal on the PATH; the window at 800x560 and 1500x900; - the retrieval manager: a build, the sources' states after files changed, an update, the three modes and the reranker, the evaluation of a six-document set, the line copied, the chat window on the index, a rebuild stopped (the index unchanged, byte for byte), the file dialogs; the window at 820x560.
The pictures of this page were taken the same way, each tool at its own size, from a folder with short paths and an empty configuration, in a network namespace of its own so that only the servers of the example are listed.
sh tools/docs/gui_shots.sh takes again, without a
person, the server manager's four pictures
(gui-servers.png, gui-servers-stop.png,
gui-servers-offload.png,
gui-servers-advanced.png) and the chat window's eighteen,
on the canary models, each under its file name. It runs the tools in
/tmp/oklm-demo (DEMO=), with a registry and
settings of their own and, where unshare -rn is allowed, in
a network namespace; the models' texts are the same from a run to the
next (a seed is set in the model settings, which the side panel shows),
the times, the pids and the free memory are the run's.
ONLY="gui-chat gui-servers-stop" takes those alone; what
the tools wrote is left in build/gui-shots. What a person's
hand did for the chat window's pictures is done by three things. The
files attached are in the repository (tests/fixtures/gui:
red-ball.png, harbour.pdf,
town.md, which tools/docs/gui_fixtures.py
makes), and the picture pasted is put on the virtual display's clipboard
by tools/gui_copy_files.py. The windows a press opens are
opened by --press (speech,
models:NAME, connect:PORT, link,
and context:N and paste for the slider and
Ctrl+V), the Connect pictures against a server and a listening agent the
script starts. The states that last a moment are caught by
--shot-when (resizing, recording,
speaking, talking, commands:/re),
which saves the window the first time it is in the state named, the
microphone being a sound file (--mic-file) and the voice a
WAV file (--speak-file); a state that never comes is said
on the error stream and the picture is not counted. The script's header
lists each picture with its model and how it is taken; none is left to a
person, and a picture is skipped with a line where its model,
ImageMagick (the two panels of gui-chat-context.png side by
side) or PyGObject with GTK 3 (the clipboard) is missing.