Command reference
Every command of oklm and every tool of
oklm-gui on one page: its synopsis, what it does, what each
option means, and the page that describes it in depth. The synopses are
the program's own usage text (oklm help,
oklm-gui --tool NAME --help), copied here by
python3 tools/docs/interfaces.py synopsis;
make docs-lint fails when one of them is no longer what the
sources print, or when a command has none.
Status: maintained; the synopses are generated from
apps/oklm.c and apps/gui/*.c, the rest is
written by hand.
Reading a synopsis
<name>is a value to give,[...]is optional,a|bis a choice, and...after an item means it may be repeated.- A number written after an option (
[-n 128],[--port 7430]) is its default. - Exit status: 0 on success, 1 on a failure, 2 on a usage error (the command's synopsis is then printed on the error stream); the commands with other statuses say so below.
oklm help(also--help,-h) prints the whole usage text;oklm transcribe --helpandoklm speak --helpprint their own.- The environment variables are in build.md.
| Area | Commands |
|---|---|
| The program | version, devices, types,
help |
| Files | info, convert, quantize,
imatrix, split, merge,
lora-merge, expert-pack,
tokenize, train-tokenizer,
ingest |
| Models and memory | models, plan, tune |
| Inference | run, chat, ppl,
embed, rerank |
| Training | train, rl, abliterate,
refusals |
| Serving | serve |
| Agent | agent, permissions,
audit |
| Retrieval | rag index, rag search,
rag info, rag eval |
| Multimodal and speech | describe, transcribe,
speak |
| Graphical tools | oklm-gui --tool launcher, chat,
models, servers, inspect,
convert, train, sessions,
rag |
Shared options
These options mean the same thing in every command that takes them. The commands below name the groups they take and describe only what is their own.
The model
| Option | Meaning |
|---|---|
-m MODEL, --model MODEL |
the model: a GGUF file (any shard of a split model opens the whole
model), a Hugging Face checkpoint directory where the synopsis says
hf_dir, or the name of an installed catalogue model (models.md) |
--mmproj FILE |
a projector (vision or audio encoder) beside the model; a catalogue name works too, and a catalogue model that carries a projector gets it without the option where the command says so |
--lora ADAPTER, --lora-scale S |
a LoRA adapter merged into the model as it loads, times S (1)
(run, chat, ppl,
serve; training.md) |
--chat-template FILE|NAME |
the chat template used in place of the file's: a Jinja file, or a
built-in format's name (vicuna, chatml,
llama2, llama3, mistral,
mistral-v3, gemma, phi3,
alpaca, deepseek-coder,
foundation-sec) (run, chat,
serve, agent, describe; inference.md) |
--template-vars JSON |
an object of more variables given to the chat template, such as
{"enable_thinking": false} |
Everywhere a model file is expected (-m,
--model, --mmproj, --draft,
--embed-model, --rerank-model,
--transcribe-model, --speak-model) the name of
an installed catalogue model can be given.
Where it computes
| Option | Meaning |
|---|---|
--backend cpu|opencl|vulkan|auto |
the compute backend; auto takes a device when there is
one. The training commands take auto|cpu|opencl|rocm
instead (opencl.md, vulkan.md, rocm_spec.md) |
--gpu-layers N, -ngl N |
on a device backend, the model's last N layers and its output head on the device, the others on the CPU; without it, as many as fit in the device's memory |
--gpu-stream N |
with --gpu-layers below the layer count: batches of N
tokens or more stream the other layers to the device (run,
chat, ppl; plan counts it) |
--devices N |
the device layers spread over N devices of the backend (0, the
default: every device of OKLM_CL_DEVICES /
OKLM_VK_DEVICES) |
-c CTX |
the context length in tokens; without it 4096, or the model's
trained context when that is smaller (agent: 32768, capped
the same way) (models.md) |
-ctk TYPE, -ctv TYPE,
--cache-type-k, --cache-type-v |
the element type of the KV cache's keys and values: f16
(default), f32, bf16, q8_0,
q4_0, q4_1, q5_0,
q5_1, iq4_nl |
--tune |
(run) tune the CPU kernels' work split first when the
cache has nothing for this model (oklm tune) |
Sampling
run, chat, serve,
describe and agent share these; in
serve they are the defaults of what a request leaves out
(inference.md, server.md).
| Option | Default | Meaning |
|---|---|---|
-n N |
128 (run), 256 (describe); none in
serve |
tokens generated at most. In serve: the length of a
request that gives none (max_tokens,
max_completion_tokens), which otherwise runs until its
sequence is full |
--temp T |
0.8 | temperature; 0 is greedy |
--top-k K |
40 | only the K likeliest tokens are kept |
--top-p P |
0.95 | the likeliest tokens whose probabilities sum to P are kept |
--min-p P |
0.05 | tokens below P times the likeliest one's probability are dropped |
--repeat-penalty R |
1 | penalty on tokens already written; 1 is off |
--seed S |
1234; in serve, a new one each request, from the
clock |
the sampler's seed. In serve: the seed of every request
that names none, the same for each, so that the same request gives the
same text |
Mixture-of-experts models
run, chat, ppl and
serve take these for a MoE model (moe.md); plan takes
--n-cpu-moe.
| Option | Meaning |
|---|---|
--expert-cache GB|auto |
(CPU) the routed experts read from the file into a per-layer LRU
cache of that size instead of being held resident; auto:
the largest that fits beside the dense part |
--pin-gb GB |
that much of the cache given to the most routed experts of the usage file, never evicted |
--expert-usage FILE |
the routing counts file (default
~/.cache/oklm/usage/<hash>.json) |
--expert-direct, --no-expert-direct |
O_DIRECT reads of the experts, or buffered ones |
--expert-loaders N |
threads that read the missing experts while the resident ones compute (4; 0: none) |
--expert-predict last|router |
the next layer's experts read ahead: its last routing, or its router applied to the current state |
--expert-pack FILE |
each missing expert read in one read from an expert pack (oklm expert-pack) |
--expert-mirror FILE |
every other read sent to a copy of the experts' file on another drive |
--expert-stats |
hit rates and reads printed at exit |
--n-cpu-moe N|all, -ncmoe |
(device backend) the routed experts of the first N layers, or of all, stay in host memory and run on the CPU, the rest of those layers on the device |
--expert-gpu GB|auto |
(device backend) a device tier of that size holding the hottest
experts of the --n-cpu-moe layers |
--expert-loaders, --expert-predict and
--expert-direct, when not given, take what
oklm tune --expert-cache kept for this model, cache size
and disk.
Speculative decoding
run and serve (inference.md). The
answers are those of plain decoding; only the speed changes.
| Option | Meaning |
|---|---|
--draft MODEL, -md |
a smaller model of the same vocabulary proposes tokens,
--draft-n N (5) a step at most |
--mtp, --mtp-model FILE |
(run) the model's own multi-token prediction block as
the draft, from the model's file or from another file |
--spec-ngram K |
prompt lookup: up to K tokens that followed the last tokens earlier in the text |
--spec-grammar |
the text a grammar forces (a key, the fixed part of a tool call) proposed as drafts |
--spec-guard RATE |
with --expert-cache: no guessed drafts while the expert
cache's recent hit rate is below RATE |
Retrieval in generation
run, chat and serve answer
from a retrieval index with --rag (rag.md); the agent has
its own options, below.
| Option | Default | Meaning |
|---|---|---|
--rag INDEX |
none | an index made by oklm rag index: each question is
searched in it and answered from the passages found |
--rag-k N |
5 | passages kept |
--rag-mode dense|sparse|hybrid |
hybrid | vectors, keywords (BM25), or both fused |
--rag-crag on|off |
on | corrective retrieval: each passage judged, the question rewritten when none is relevant |
--rag-route on|off |
off | the model decides between no retrieval, one search and several |
--rag-rewrite on|off |
on | (chat, serve) a follow-up rewritten as a
standalone query before it is searched |
--rag-followup rewrite|join|both |
both | (chat, serve) what a follow-up searches:
the rewrite, the previous question joined with it, or both |
--rag-rerank MODEL,
--rag-rerank-top N |
none, 20 | the candidates reordered by a reranker before the best are kept |
--rag-graph on|off |
off | the index's graph walked and fused with the search (an index built
with --graph) |
--rag-critique on|flag|off |
off | the answer checked against its sources: on generates it
once more when a cited statement is unsupported, flag only
flags it |
--rag-log FILE |
none | every retrieval appended to FILE as a JSON line |
Output for programs
| Option | Meaning |
|---|---|
--json |
the result as one JSON object instead of text, where a command lists it |
--progress-json FILE|- |
the run as JSON lines, one event a line, in FILE; - is
standard output, the human report then going to standard error
(convert, quantize, ppl,
train, rl, abliterate,
refusals, rag index; training.md, convert.md) |
--dry-run |
(convert, quantize) say the size the file
would have, and write nothing |
Other names
Options spelled as other programs spell them are read as OKLM's:
| Other name | Read as |
|---|---|
--model |
-m |
-ngl |
--gpu-layers |
-np |
--parallel |
-md |
--draft |
-ncmoe |
--n-cpu-moe |
--cache-type-k, --cache-type-v |
-ctk, -ctv |
--system-prompt, --permission-mode
(agent) |
--system, --mode |
--allowed-tools, --disallowed-tools
(agent) |
--allowedTools, --disallowedTools |
The program
oklm version
oklm devices
oklm types
oklm help
version prints the library's version.
devices lists the compute backends built in and found, the
CPU's features and the thread count. types lists the tensor
element types and what OKLM can do with each (read, compute, write).
help prints the usage text. None takes an option. See build.md and core.md.
Files
oklm info
oklm info [-t] [--json] <model.gguf>
Prints a model file's metadata and the compatibility verdict: whether OKLM runs it, and why not when it does not (convert.md).
| Option | Meaning |
|---|---|
-t |
every tensor too: name, type, shape |
--json |
one object, with the layout by block |
oklm convert
oklm convert <hf_dir> --out <model.gguf> [--type f32|f16|bf16|q8_0|q4_0|q4_1|q5_0|q5_1] [--name NAME] [-v] [--dry-run]
Converts a Hugging Face checkpoint (config.json and
safetensors) to a GGUF file (convert.md).
| Option | Meaning |
|---|---|
--out FILE |
the GGUF file written (required) |
--type T |
the matrices' type: f32, f16 (default),
bf16, q8_0, q4_0,
q4_1, q5_0, q5_1 |
--name NAME |
general.name (default: the directory's name) |
-v |
each tensor as it is written |
--dry-run, --progress-json FILE|- |
Output for programs |
oklm quantize
oklm quantize [--pure] [--imatrix FILE] [--override-kv KEY=TYPE:VALUE] [--dry-run] <in.gguf> <out.gguf> <type>
Requantizes a GGUF model to a llama-quantize type
(Q4_K_M, Q8_0, IQ4_XS...; the
usage error lists them all) (convert.md).
| Option | Meaning |
|---|---|
--pure |
every matrix in the base type, none kept at a larger one |
--imatrix FILE |
an importance matrix (oklm imatrix) that weighs the
rounding error |
--override-kv KEY=TYPE:VALUE |
a metadata key set in the result (TYPE str,
int, float or bool;
repeatable) |
--dry-run, --progress-json FILE|- |
Output for programs |
oklm imatrix
oklm imatrix -m <model> -f <text.txt> -o <out.imatrix.gguf> [-c 512] [--chunks N] [--process-output]
Computes an importance matrix for
oklm quantize --imatrix from a calibration text, in
llama-imatrix's format (convert.md).
| Option | Meaning |
|---|---|
-f FILE |
the calibration text |
-o FILE |
the importance matrix written |
-c N |
the length of each chunk in tokens (512) |
--chunks N |
chunks read at most (default: the whole text) |
--process-output |
the output matrix collected too |
oklm split, oklm merge
oklm split <in.gguf> <out-prefix> --split-max-size N[K|M|G] | --split-max-tensors N
oklm merge <shard.gguf> <out.gguf>
split writes a model as shards
<out-prefix>-0000i-of-0000N.gguf; merge
joins the shards of a split model, given any one of them, into one file
(convert.md).
| Option | Meaning |
|---|---|
--split-max-size N[K|M|G] |
a shard's largest size |
--split-max-tensors N |
a shard's largest tensor count |
oklm lora-merge
oklm lora-merge <model.gguf> <adapter.gguf> <out.gguf> [--scale 1] [--save-type bf16]
Merges a LoRA adapter (one oklm train --lora wrote, or
llama.cpp's format) into the model: W + scale x alpha / rank x B A (training.md).
| Option | Meaning |
|---|---|
--scale S |
the adapter's weight (1) |
--save-type bf16 |
the merged weights and the other F32 / F16 matrices written in BF16 |
oklm expert-pack
oklm expert-pack -m <model.gguf> [-o FILE]
Writes an expert pack of a MoE model: each routed expert's three
matrices in one aligned record, which --expert-pack FILE
reads in one read (moe.md).
-o FILE names the pack (default
<model.gguf>.experts).
oklm tokenize
oklm tokenize --model <model.gguf|tokenizer.json|dir> [--bos] [--no-special] [--pieces] <text>
oklm tokenize --model <...> --decode <id,id,...>
Prints the token ids of a text, or the text of token ids, with the
tokenizer of a model file, a tokenizer.json or a checkpoint
directory (tokenizer.md).
| Option | Meaning |
|---|---|
--bos |
the tokenizer's special tokens added (BOS, EOS) as the model expects them |
--no-special |
special tokens written in the text are not recognized, they are tokenized as text |
--pieces |
each id with its text |
--decode id,id,... |
the other way: the text of those ids |
oklm train-tokenizer
oklm train-tokenizer --vocab <N> --out <tok.gguf> [--min-freq F] [--pre gpt-2|llama-bpe|qwen2|smollm]
[--special TEXT]... [--chat-template FILE] [-v] <files...>
Trains a byte-level BPE tokenizer on text files and writes it as a vocabulary-only GGUF (tokenizer.md).
| Option | Meaning |
|---|---|
--vocab N |
the vocabulary's size, special tokens included |
--out FILE |
the file written |
--min-freq F |
the merges stop when the most frequent pair occurs fewer than F times (2) |
--pre NAME |
the pre-tokenizer's pattern set: gpt-2 (default),
llama-bpe, qwen2, smollm |
--special TEXT |
a special token (repeatable, 16 at most) |
--chat-template FILE |
a Jinja chat template kept in the file |
-v |
a log line every 500 merges |
oklm ingest
oklm ingest <document>...
Prints documents as the text a model reads: text, Markdown, CSV, JSON
indented, HTML rendered, DOCX, PDF (multimodal.md). It takes no option;
oklm run --doc FILE puts the same text before a prompt.
Models and memory
oklm models
oklm models [list] [--category chat|code|vision|audio|speech|security|retrieval] [--installed] [--json]
oklm models pull NAME... [--accept-licence]
oklm models rm NAME... | models path NAME [--mmproj] | models dir
The catalogue of models OKLM offers to download (it ships none):
list shows them, pull downloads,
rm removes, path prints an installed model's
file and dir the model directory (models.md).
| Option | Meaning |
|---|---|
--category C |
(list) one category: chat,
code, vision, audio,
speech, security, retrieval |
--installed |
(list) the installed models only |
--json |
(list) the list as JSON |
--accept-licence |
(pull) answers yes to a licence that has to be agreed
to |
--mmproj |
(path) the model's projector file instead of the
model's |
oklm plan
oklm plan -m <model.gguf> [-c CTX] [--parallel N] [-ctk TYPE] [-ctv TYPE] [--gpu-layers N] [--n-cpu-moe N|all] [--backend cpu|opencl|vulkan|auto]
[--draft MODEL] [--mmproj FILE] [--gpu-stream N] [--json] (aliases: --model, -np, -ngl, -ncmoe, -md)
[--serve [--kv-mem MB] [--tiles SIDE] [--transcribe-model DIR [--transcribe-backend cpu|opencl|vulkan|auto]]
[--speak-model FILE [--speak-vocoder FILE] [--speak-backend cpu|opencl|vulkan|auto]]
[--embed-model FILE [--embed-backend B]] [--rerank-model FILE [--rerank-backend B]]]
oklm plan --list [DIR] [--recommend] [-c CTX] [--backend cpu|opencl] [--n-cpu-moe N|all] [--json]
Says the memory a model would take, from its headers alone, against
the available RAM and the device's free memory: fits, fits with N layers
on the device, or does not fit (exit status 0 when it fits, 3 when it
does not). plan --list does it for every GGUF model under
models/ or DIR (memory-plan.md).
| Option | Meaning |
|---|---|
-c, -ctk, -ctv,
--backend, --gpu-layers,
--gpu-stream, --n-cpu-moe |
what the run would be given (Where it computes) |
--parallel N, -np |
the sequences a server would open |
--serve |
project the model as oklm serve runs it: one cache
shared by the requests, what is left beside the model on a device, four
contexts on the CPU; the verdict then also says
with a smaller cache (one context at least, less than
four), and does not fit when the cache holds less than one context (memory-plan.md) |
--kv-mem MB |
with --serve: the size of that cache, as
oklm serve --kv-mem takes it (-1: the whole
context for each sequence) |
--draft MODEL, --mmproj FILE |
a draft model and a projector counted beside the model |
--tiles SIDE, --transcribe-model DIR,
--transcribe-backend B, --speak-model FILE,
--speak-vocoder FILE, --speak-backend B,
--embed-model FILE, --embed-backend B,
--rerank-model FILE, --rerank-backend B |
with --serve: what the server loads beside the model,
as oklm serve takes these options, each part counted where
the server places it and taken out of the cache's room first (memory-plan.md) |
--list [DIR] |
every model under models/ or DIR, with its need at 8192
tokens and its verdict |
--recommend |
(--list) names the largest model that fits resident and
the largest that needs the expert cache |
--json |
one object, with each layer's placement (memory-plan.md) |
oklm tune
oklm tune -m <model.gguf|hf_dir> [--backend cpu|opencl|vulkan|auto] [--budget S] [-n TOKENS] [--force] [--cache-dir DIR]
[--expert-cache GB|auto | --streaming] [--gen TOKENS]
Tunes the CPU kernels' work split for a model on this machine and
keeps the result in a cache that contexts apply
(OKLM_TUNE=0 turns that off). With
--expert-cache it tunes a MoE model's streaming settings
instead (moe.md, kernel-autotune-plan.md).
| Option | Meaning |
|---|---|
--budget S |
seconds the search may take (60); the best found so far is kept |
-n TOKENS |
prompt tokens timed per run (512; 256 for the streaming settings) |
--gen TOKENS |
(--expert-cache) the tokens decoded in each timed run
(64) |
--force |
tune again although the cache has a result |
--cache-dir DIR |
where the results are kept (default
$XDG_CACHE_HOME/oklm/tuning, else
~/.cache/oklm/tuning) |
--expert-cache GB|auto, --streaming |
time --expert-loaders, --expert-predict
and --expert-direct with that cache
(--streaming: auto) and keep the fastest with
the same output |
Inference
oklm run
oklm run -m <model.gguf|hf_dir> -p <prompt> [--doc FILE] [-n 128] [--temp 0.8] [--top-k 40] [--top-p 0.95] [--min-p 0.05] [--repeat-penalty 1]
[--seed S] [-c CTX] [--backend cpu|opencl|vulkan|auto] [--gpu-layers N] [--tune]
[--grammar FILE.gbnf | --json-schema FILE.json | --json] [--draft MODEL | --mtp | --mtp-model FILE | --spec-ngram K]
[--draft-n 5] [--spec-grammar] [--spec-guard RATE] [--observe FILE [--observe-attn all|L,L...] [--observe-lens K]]
[--ablate L:attn|L:ffn|L:head:H[,...] | --ablate-scan blocks|heads:L] [--ablate-mean]
[--lora ADAPTER [--lora-scale 1]] (also chat, ppl, serve: a LoRA adapter merged in as the model loads)
Completes a prompt and prints the text (inference.md). It takes the model, placement, sampling, MoE, speculative decoding and retrieval options, and:
| Option | Meaning |
|---|---|
-p PROMPT |
the prompt |
--doc FILE |
a document whose text goes before the prompt, read as
oklm ingest reads it |
--grammar FILE.gbnf |
the output constrained by a GBNF grammar (inference.md) |
--json-schema FILE.json |
the output constrained to the values of a JSON schema |
--json |
the output constrained to any JSON value |
--observe FILE |
a JSON object a line for the prompt's last token then each generated one: what each block wrote, the experts picked (observe.md) |
--observe-attn all|L,L... |
with it, the attention of the last token over the positions, for every block or those listed |
--observe-lens K |
with it, the K likeliest tokens the output head gives after each block |
--ablate L:attn|L:ffn|L:head:H[,...] |
those parts taken out of the run, and what that does to the next token said (observe.md) |
--ablate-scan blocks|heads:L |
every block's two parts, or every head of block L, taken out in turn, a line each |
--ablate-mean |
a part taken out is replaced by its mean over the prompt, not by zero |
oklm chat
oklm chat (-m <model.gguf|hf_dir> | --url URL [--api-key KEY] [--remote-model NAME]) [--system TEXT] [--chat-template FILE|NAME] [--template-vars JSON]
[--tools FILE] [--tool-choice auto|required]
[--tool-format hermes|llama3|mistral|glm45|glm4|harmony|lfm2|gemma4] [--temp T] [--top-k K] [--top-p P] [--min-p P] [--repeat-penalty R]
[-c CTX] [--backend cpu|opencl|vulkan|auto] [--gpu-layers N] [--resume] [--kv-save] [--chat-file FILE] [--stats] [--tui]
[--mmproj FILE|NAME|none [--tiles SIDE] [--fps 1] [--frames 8] [--ffmpeg]]
[--auto-compact on|off] [--auto-continue on|off] [--continue-max 8]
An interactive chat on the terminal, with a model in this process
(-m) or a server's (--url). A line starting
with / is a command (/help lists them:
/system, /reset, /model,
/context, /attach, /save,
/export, /tui, /exit...; server.md has the table shared
with the window and the dashboard). It takes the
model, placement, sampling, MoE and retrieval options, and (inference.md):
| Option | Meaning |
|---|---|
--url URL |
chat with the model of an OpenAI-compatible server
(oklm serve, llama.cpp's server, another) in place of one
in this process: http://host:port or
https://host:port, with or without /v1; any
other path is the API's base and stands for /v1 and an
endpoint given whole (.../v1/chat/completions) is read as
its base; OpenAI, Gemini, Anthropic, Mistral, DeepSeek and Ollama are
reached that way (gui.md). Not with
-m. Without -m and --url, the
baseUrl setting is the server. What each command and option
does there, and which need -m: inference.md |
--api-key KEY |
with --url: the key sent
(Authorization: Bearer); else OKLM_API_KEY,
else OPENAI_API_KEY |
--remote-model NAME |
with --url: the server's model. Without it, the one the
server lists (GET /v1/models); among several, a list to
choose from on a terminal, and status 2 with the list when the input is
not typed |
--system TEXT |
the system prompt |
--tools FILE |
a JSON file of tool definitions the model may call;
/tool RESULT answers a call (inference.md) |
--tool-choice auto|required |
whether the model may answer in text or must call a tool |
--tool-format NAME |
the wire format of tool calls, in place of the template's:
hermes, llama3, mistral,
glm45, glm4, harmony,
lfm2, gemma4 |
--resume |
go on with the conversation kept in
.oklm/chat/last.json |
--chat-file FILE |
keep the conversation in that file instead |
--kv-save |
keep the cache beside the conversation (.kv), so
--resume starts warm (inference.md) |
--auto-compact on|off |
near the context's end the older messages are summarized into a note
by themselves (on), or only with /compact; for this run,
over autoCompact of the user's settings, which
/autocompact writes and oklm agent shares (inference.md) |
--auto-continue on|off |
a reply the context or the token limit cut is gone on with by itself
(on), or only with /continue; for this run, over
autoContinue |
--continue-max N |
the pieces a reply is continued with in a row at most (8); for this
run, over continueMax |
--stats |
each turn's cached and evaluated tokens |
--tui |
the full-screen UI: Markdown, folded reasoning, a status bar |
--mmproj FILE|NAME|none |
the projector: /attach FILE then takes a picture, a
sound or a video; none loads none (multimodal.md) |
--tiles SIDE, --fps F,
--frames N, --ffmpeg |
how attached pictures and videos are read, as oklm describe |
--observe FILE |
as oklm run |
Without --resume, --kv-save or
--chat-file nothing is written.
oklm ppl
oklm ppl -m <model.gguf> -f <text.txt> [-c CTX] [--progress-json FILE|-]
Perplexity of a model over a text file, the second half of each
window scored (inference.md). It
takes placement, MoE and --lora
options, -f FILE (the text) and
--progress-json.
oklm embed
oklm embed -m <model.gguf|hf_dir> [--json] [--query] [--pooling cls|mean|last] [--no-normalize] [--query-prefix TEXT]
[--passage-prefix TEXT] [-c CTX] [-b BATCH] [--backend cpu|opencl|vulkan|auto] [--gpu-layers N] <text>... | --file F
One embedding a text, or a line of --file F, from a text
encoder or a decoder embedding model: one vector a line (rag.md).
| Option | Meaning |
|---|---|
--json |
OpenAI's /v1/embeddings response |
--query |
the texts are queries: the query prefix is put before them, not the passage prefix |
--pooling cls|mean|last |
how the tokens' vectors become one, in place of what the model says |
--no-normalize |
the vectors not L2-normalized |
--query-prefix TEXT,
--passage-prefix TEXT |
the text put before a query or a passage, in place of the model's |
-b BATCH |
texts evaluated at once |
--file F |
the texts, one a line |
oklm rerank
oklm rerank -m <model.gguf|hf_dir> -q QUERY [--top N] [--json] [-c CTX] [--backend cpu|opencl|vulkan|auto] <passage>... | --file F
Scores passages against a query and prints them best first: a cross-encoder gives its relevance logit, a chat model answers whether the passage is relevant (rag.md).
| Option | Meaning |
|---|---|
-q QUERY, --query QUERY |
the query |
--top N |
the N best only |
--json |
/v1/rerank's response |
--file F |
the passages, one a line |
Training
oklm train
oklm train (-f <text.txt> | --chat-data <conv.jsonl>) --out <prefix> [-m <model> [--lora R ...] | --resume <ckpt.gguf> | --tokenizer <tok> | --vocab N]
[--arch gpt2|llama|qwen2|qwen3] [--layers 4] [--embd 128] [--heads 4] [--kv-heads H] [--head-dim D] [--ff N]
[--rope-theta 10000] [--ctx 256]
[--steps 1000] [--batch 4] [--accum 1] [--seq T] [--lr 3e-4] [--lr-min 3e-5] [--warmup 50] [--wd 0.1] [--clip 1] [--dropout 0] [--seed S]
[--val-frac 0.1] [--eval-every 100] [--eval-batches 8] [--save-every N] [--sample-every N] [--sample-len 200]
[--backend auto|cpu|opencl|rocm] [--recompute]
[--quantized-base] [--act-type f32|bf16] [--opt-type f32|bf16] [--save-type f32|bf16] [--matmul-precision f32|bf16|f16]
[--grad-type f32|bf16] [--loss-scale dynamic|N|off]
Trains a GPT-2, Llama or Qwen model from scratch, fine-tunes a model
(-m), trains LoRA adapters on it (--lora), or
continues a checkpoint (--resume). Checkpoints are
<prefix>-<step>.gguf with a .train
sidecar of the optimizer's state (training.md).
| Option | Default | Meaning |
|---|---|---|
-f FILE |
the training text | |
--chat-data FILE |
conversations as JSON lines instead, the loss on the assistant turns (training.md) | |
--out PREFIX |
the checkpoints' prefix; its directories are made when missing | |
-m MODEL |
the model fine-tuned | |
--resume FILE |
the checkpoint (or with -m, the adapter) continued; its
options come from the sidecar |
|
--tokenizer FILE, --vocab N |
from scratch: a tokenizer file, or a BPE tokenizer of N tokens trained on the text | |
--arch gpt2|llama|qwen2|qwen3 |
gpt2 | from scratch: the architecture |
--layers, --embd, --heads,
--ctx |
4, 128, 4, 256 | blocks, width, attention heads, trained context |
--kv-heads H, --head-dim D,
--ff N, --rope-theta T |
Llama and Qwen: key / value heads, a head's width, the feed-forward width, RoPE's base (10000) | |
--steps N |
1000 | updates |
--batch N, --seq T,
--accum N |
4, , 1 | rows a batch, tokens a row, batches summed per update |
--lr, --lr-min, --warmup |
3e-4, 3e-5, 50 | the learning rate, its floor at the end of the cosine decay, the warm-up steps |
--wd, --clip, --dropout |
0.1, 1, 0 | weight decay, the gradient's norm limit, residual dropout |
--seed S |
the seed of the initial weights and of the batches | |
--val-frac F, --eval-every N,
--eval-batches N |
0.1, 100, 8 | the text's last share kept for validation, how often and over how many windows its loss is measured |
--save-every N |
a checkpoint every N steps (and one at the end) | |
--sample-every N, --sample-len L |
off, 200 | every N steps and at the end, a text of L tokens the model writes from its end-of-text token |
--lora R, --lora-alpha A,
--lora-targets LIST |
rank-R adapters on -m's projections
(q,k,v,o,gate,up,down, or all), the model
frozen; the checkpoints are adapter files |
|
--quantized-base |
with --lora: the frozen matrices kept in
-m's types instead of F32 |
|
--recompute |
each block's activations recomputed in the backward pass: far less activation memory, about one more forward pass | |
--act-type, --opt-type,
--grad-type f32|bf16 |
f32 | the stored activations, AdamW's moments, the gradients held between passes, in BF16 to halve their memory (training.md) |
--save-type f32|bf16 |
f32 | the checkpoints' matrices |
--matmul-precision f32|bf16|f16 |
f32 | (--backend rocm) the inputs of the large products
rounded to that type |
--loss-scale dynamic|N|off |
off | the gradients computed on the loss times a scale and divided back, for F16 products |
--backend, --progress-json |
Where it computes, Output for programs |
oklm rl
oklm rl (-m <model> | --resume <ckpt.gguf>) -f <tasks.jsonl> --out <prefix> [--steps 100] [--tasks 8] [--group 8]
[--max-new 32] [--temp 1] [--lr 1e-5] [--wd 0] [--val-frac 0.1] [--eval-every 20] [--save-every N] [--seed S]
[--updates 1] [--clip 0.2] [--kl 0] [--ref <model>] [--backend auto|cpu|opencl|rocm]
Reinforcement-learning fine-tuning on tasks with known answers
(tasks.jsonl: one
{"prompt": ..., "answer": ...} a line). A completion earns
1 when its last number is a numeric answer, or when it starts with any
other answer; the advantages are relative to the group (training.md).
| Option | Default | Meaning |
|---|---|---|
-m MODEL, --resume FILE |
the model, or a checkpoint continued | |
-f FILE, --out PREFIX |
the tasks, the checkpoints' prefix | |
--steps N, --tasks N,
--group N |
100, 8, 8 | updates, tasks a step, completions sampled per task |
--max-new N, --temp T |
32, 1 | a completion's length and sampling temperature |
--lr, --wd, --seed |
1e-5, 0 | as oklm train |
--val-frac F, --eval-every N,
--save-every N |
0.1, 20 | the tasks held out, how often they are scored, how often a checkpoint is written |
--updates N, --clip C |
1, 0.2 | updates from each sampled batch, the ratio clipped to 1 +/- C past the first |
--kl W, --ref MODEL |
0 | the weight of a KL penalty against a reference model (default: the starting model) |
oklm abliterate
oklm abliterate -m <model> --harmful <refused.txt> --harmless <answered.txt> [--out <out.gguf>] [--layer auto|N]
[--band 0.3,0.7] [--scale 1] [-n PROMPTS] [--system TEXT] [--template-vars JSON] [--save-dirs FILE]
oklm abliterate -m <model> --load-dirs FILE --layer N --out <out.gguf> [--scale 1]
Finds a chat model's refusal direction per layer from two prompt sets
and, with --out, projects it out of the weights (training.md).
| Option | Default | Meaning |
|---|---|---|
--harmful FILE, --harmless FILE |
prompts the model declines, prompts it answers, one a line | |
--out FILE |
the model written; without it the directions are only measured | |
--layer auto|N |
auto | the layer whose direction is removed; auto: the best
within --band |
--band LOW,HIGH |
0.3,0.7 | the share of the depth auto looks in |
--scale S |
1 | how much of the direction is removed (0 to 2) |
-n PROMPTS |
all | the first PROMPTS lines of each file only |
--system TEXT, --template-vars JSON |
the system prompt and the template variables the prompts are rendered with | |
--save-dirs FILE, --load-dirs FILE |
keep the directions; write another layer or scale from them without recomputing |
oklm refusals
oklm refusals -m <model> [--prompts <refused.txt>] [--ref <model> --harmless <answered.txt>] [-n 48] [--limit N]
[--backend cpu|opencl|vulkan|auto] [--system TEXT] [--template-vars JSON] [--show]
Answers each prompt of a file greedily and counts the answers that read as refusals; with a reference model, the first-token KL divergence from it on prompts the model should answer (training.md).
| Option | Default | Meaning |
|---|---|---|
--prompts FILE |
the prompts judged | |
--ref MODEL --harmless FILE |
the reference model and the prompts the divergence is measured on | |
-n N |
48 | tokens of each answer |
--limit N |
all | the first N prompts only |
--show |
each answer printed as it is judged (REFUSED or
answered) |
|
--system TEXT, --template-vars JSON |
as oklm abliterate |
Serving
oklm serve
oklm serve -m <model> [--host 127.0.0.1] [--port 7430] [--tls-cert FILE --tls-key FILE] [--api-keys FILE] [--insecure] [--fixed-context] [--fixed-model]
[--parallel N] [--queue N] [--workers N] [--kv-mem MB] [--prefill-chunk N] [--prefill-share F] [--deterministic] [--observe] [--observe-lens K] [--spec-ngram K] [--model-id NAME] [-c 4096] [--backend cpu|opencl|vulkan|auto] [--temp T] [--top-k K] [--top-p P]
[--min-p P] [--repeat-penalty R] [--seed S] [-n N]
[--mmproj FILE [--tiles SIDE] [--ffmpeg]] [--draft MODEL [--draft-n 5]] [--swap-mem MB] [--lora FILE [--lora-scale S]] [--chat-template FILE|NAME]
[--lora-module NAME=ADAPTER.gguf ...] [--spec-grammar] [--spec-guard RATE] [--prefix-dir DIR [--prefix-min N] [--prefix-disk-mb MB]]
[--embed-model EMB [--embed-model-id NAME] [--embed-backend cpu|opencl|vulkan|auto]]
[--rerank-model R [--rerank-model-id NAME] [--rerank-backend cpu|opencl|vulkan|auto]]
[--transcribe-model W [--transcribe-model-id NAME] [--transcribe-backend BACKEND]] [--upload-max MB] [--ingest-keep SECONDS]
[--speak-model S [--speak-vocoder FILE] [--speak-voice FILE] [--speak-voices DIRS] [--speak-model-id NAME] [--speak-backend BACKEND] [--speak-max BYTES]]
[--preload]
An HTTP server: OpenAI-compatible and Anthropic-compatible routes, a
dashboard on /, requests sharing one context in batches.
Beyond loopback, TLS and API keys are required. server.md has every option with its default (Running it) and every route (Endpoints). Besides the model, placement, sampling, MoE, speculative decoding and retrieval options:
| Option | Meaning |
|---|---|
--host ADDR, --port N |
where it listens (127.0.0.1, 7430) |
--tls-cert FILE --tls-key FILE |
HTTPS |
--api-keys FILE |
the accepted keys, one a line |
--insecure |
a non-loopback address allowed without TLS or keys |
--fixed-context, --fixed-model |
POST /api/context and POST /api/model
refused |
--parallel N, --queue N,
--workers N |
requests generating at once, requests waiting, connections served |
--kv-mem MB |
the paged KV cache's size (0: what fits; -1: the contiguous cache) |
--swap-mem MB |
host memory for the rows of preempted requests |
--prefill-chunk N, --prefill-share F |
prompt tokens of one request evaluated in a step, and their share of a step while others decode |
--deterministic |
the same tokens at temperature 0 whether the prompt was cached or not (server.md) |
--prefix-dir DIR, --prefix-min N,
--prefix-disk-mb MB |
the prefix cache kept on disk between runs (server.md) |
--observe, --observe-lens K |
activations recorded, read with GET /api/activations
(server.md) |
--model-id NAME |
the model's name in the API |
--lora-module NAME=ADAPTER.gguf |
an adapter served beside the model, used by the requests whose
model is NAME (repeatable) |
--mmproj FILE, --tiles SIDE,
--ffmpeg |
chat messages take images, videos and sounds (server.md) |
--embed-model EMB, --embed-model-id NAME,
--embed-backend B |
/v1/embeddings (server.md) |
--rerank-model R, --rerank-model-id NAME,
--rerank-backend B |
/v1/rerank (server.md) |
--transcribe-model W,
--transcribe-model-id NAME,
--transcribe-backend B |
/v1/audio/transcriptions and
/v1/audio/translations (server.md) |
--speak-model S, --speak-vocoder FILE,
--speak-voice FILE, --speak-voices DIRS,
--speak-model-id NAME, --speak-backend B,
--speak-max BYTES |
/v1/audio/speech and /v1/audio/voices (server.md) |
--preload |
the transcription and speech models loaded at start, not by the first request that needs them |
--upload-max MB,
--ingest-keep SECONDS |
the largest document or sound sent as a body (32), how long a conversation's documents are kept (3600) (server.md) |
--rag-dir DIR |
a directory of indexes a request may name (server.md) |
oklm serve --list, --stop
oklm serve --list [--json] | --stop PORT|PID
The servers of this user that run (each registers itself at start:
pid, port, model, command), --json as JSON;
--stop ends one by its port or its pid (server.md).
oklm serve --keep
oklm serve --keep NAME -m <model> [the options of serve] | --rekeep NAME ... | --kept [--json] | --unkeep NAME
A server kept across logins, as a service of the user (server.md).
| Option | Meaning |
|---|---|
--keep NAME ... |
the command that follows kept under NAME and started now, at each login and again when it fails |
--rekeep NAME ... |
the command kept under NAME changed |
--kept |
the kept servers with their state (--json: as
JSON) |
--unkeep NAME |
one stopped and its unit deleted |
--start-kept NAME |
a kept server that does not run started, as its service starts it (what Windows runs at login) |
Agent
oklm agent
oklm agent (-m <model> [--chat-template FILE|NAME] | --url URL [--api-key KEY] [--remote-model NAME] | --mock FIXTURE.jsonl) [-p TASK] [--project DIR]
[--mode default|acceptEdits|plan|bypassPermissions] [--dangerously-skip-permissions] [--allow RULE] [--ask RULE] [--deny RULE]
[--max-turns 32] [--max-tokens 8192] [--max-background 2] [--no-sandbox] [--allow-network] [--keep-services] [--limit NAME=VALUE]
[--auto-compact on|off] [--auto-continue on|off] [--continue-max 8]
[--thinking hide|show] [--system TEXT] [--append-system-prompt TEXT] [--output-style default|explanatory|learning|NAME]
[--no-memory] [--save-training FILE] [--no-tool-grammar] [--debug] [--worktree [NAME]] [--temp T] [-c 32768]
[--backend cpu|opencl|vulkan|auto] [--tool-rag K] [--rag INDEX | --rag-project] [--rag-memory] [--rag-inject off|always|route]
[--resume [SESSION]] [--fork-session] [--kv-save]
[--output-format text|json|stream-json] [--input-format text|stream-json] [--listen [PORT]] [--allowedTools RULES]
[--disallowedTools RULES] [--add-dir DIR] [--mcp-config FILE [--strict-mcp-config]]
An agent that reads, searches and edits a project's files and runs
commands within permission rules: a REPL on the terminal, or with
-p one task and an exit. user-guide.md is the guide, agent-loop.md the loop and the headless use,
commands.md its slash commands. It takes the model, --backend, -c
(32768) and --temp, and:
| Option | Meaning |
|---|---|
--url URL, --api-key KEY,
--remote-model NAME |
the model of an OpenAI-compatible server instead of one in process (agent-loop.md) |
--mock FIXTURE.jsonl |
a scripted model, for tests |
-p TASK |
one task, without a terminal; the exit status is 0 when it succeeded, 1 when it ended in an error or was not answered for want of room in the context (agent-sessions.md) |
--project DIR |
the project's root (default: the current directory) |
--mode MODE |
the permission mode: default, acceptEdits,
plan, bypassPermissions (agent-permissions.md) |
--dangerously-skip-permissions |
the bypass mode |
--allow RULE, --ask RULE,
--deny RULE |
a permission rule for the session (repeatable) |
--allowedTools RULES,
--disallowedTools RULES |
several allow or deny rules in one argument, commas or blanks between them |
--add-dir DIR |
another working directory |
--max-turns N, --max-tokens N |
model turns a request may take (32), tokens of a turn (8192).
Without --max-tokens the 8192 are the size of one piece of
an answer, which goes on past them; given, they end an answer for
good |
--auto-compact on|off,
--auto-continue on|off, --continue-max N |
compaction near the window's end, and going on with an answer the
token limit or the context cut, by themselves (on, the default) or only
with /compact and /continue; the pieces in a
row at most (8). Over the settings autoCompact,
autoContinue, continueMax (agent-sessions.md) |
--max-background N |
background sub-agents at once (2) (agent-tools.md) |
--no-sandbox, --allow-network |
commands not confined; the network kept in the sandbox (agent-permissions.md) |
--limit NAME=VALUE |
a resource limit of the commands: cpu=S,
memory=8G, file=4G, files=N,
procs=N |
--keep-services |
the service jobs left running when the session ends (processes.md) |
--thinking hide|show |
the model's reasoning folded or shown live |
--system TEXT,
--append-system-prompt TEXT |
the system prompt replaced, or text added to it |
--output-style NAME |
default, explanatory,
learning, or a style of the project |
--no-memory |
the OKLM.md memory files left out (agent-settings.md) |
--save-training FILE |
the run appended as a JSONL training example |
--no-tool-grammar |
tool calls not constrained by the tools' grammar |
--debug |
the debug log written (agent-settings.md) |
--worktree [NAME] |
work in a git worktree of its own (git.md) |
--resume [SESSION], --fork-session |
resume the latest session, or the one named; as a copy in a new session (agent-sessions.md) |
--kv-save |
the in-process model's cache kept beside the transcript |
--output-format text|json|stream-json,
--input-format text|stream-json |
the headless formats (agent-loop.md) |
--listen [PORT] |
serve the agent over HTTP on loopback for other programs (agent-loop.md) |
--mcp-config FILE,
--strict-mcp-config |
MCP servers from that file; only those (agent-extensions.md) |
--tool-rag K |
the K tools, skills and sub-agent types that fit each message retrieved, the others deferred (rag.md) |
--rag INDEX, --rag-project,
--rag-memory |
the Retrieve tool over an index, over the project's documentation, over the earlier sessions too (rag.md) |
--rag-inject off|always|route |
passages given with each message, always or when the router says so |
--rag-critique on|flag|off |
the final answers checked against their sources |
oklm agent config, skills, sessions
oklm agent config get KEY | set KEY VALUE | unset KEY [--scope local|project|user] [--project DIR]
oklm agent skills validate PATH... | agent skills list [--project DIR]
oklm agent sessions [list] [--json] | export ID [--format markdown|json] [-o FILE] | rm ID [--project DIR]
config reads a setting (the value in force) or writes it
in one settings file (agent-settings.md).
skills validate checks skills as the agent loads them,
skills list lists those it finds (agent-tools.md).
sessions lists the project's sessions, writes one as
Markdown or JSON, or removes one (agent-sessions.md).
| Option | Meaning |
|---|---|
--scope local|project|user |
(config) the settings file: the project's personal one
(default), the project's, the user's |
--project DIR |
the project's root |
--json |
(sessions list) the list as JSON |
--format markdown|json, -o FILE |
(sessions export) the format, the file written |
oklm agent plugin, mcp, commands, agents
oklm agent plugin ... | mcp ... | commands [--markdown] | agents | attach ID | send ID MESSAGE | stop ID
oklm agent --stop-listening PORT
Each of these prints its own usage when given nothing.
| Command | What it does |
|---|---|
agent plugin list | install SOURCE [--scope user|project] | remove | enable | disable NAME | marketplace add NAME SOURCE | marketplace list |
plugins and marketplaces (agent-extensions.md) |
agent mcp list | get NAME | add NAME [--scope S] [--transport stdio|http|sse] [--env K=V] [--header 'K: V'] -- COMMAND | URL | add-json NAME JSON | remove NAME | login NAME | logout NAME |
MCP servers (agent-extensions.md) |
agent commands [--markdown] |
the slash-command reference; --markdown is commands.md |
agent agents | attach ID | send ID MESSAGE | stop ID |
the background sessions: listed, followed, sent a message, ended (agent-tools.md) |
agent --stop-listening PORT |
asks the agent listening on that port to end: it closes its listener and keeps its session |
oklm permissions, oklm audit
oklm permissions [--project DIR] [--mode MODE] <Tool> <argument>
oklm audit [--tool T] [--decision D] [--session S] [log.jsonl ...]
permissions says how the project's rules decide a tool
call, without running it: exit status 0 allow, 3 ask, 4 deny (agent-permissions.md).
audit lists the permission decisions recorded in
.oklm/audit (agent-permissions.md).
| Option | Meaning |
|---|---|
--project DIR, --mode MODE |
(permissions) the project and the permission mode the
call is judged in |
--tool T, --decision allow|ask|deny,
--session S |
(audit) only the decisions of that tool, of that kind,
of that session |
Retrieval
oklm rag index PATH... --out INDEX [--model <model.gguf|hf_dir>|hash[:DIM]] [--query-prefix TEXT] [--passage-prefix TEXT]
[--backend cpu|opencl|vulkan|auto] [--chunker auto|fixed|recursive|code|semantic] [--chunk-size N] [--overlap N]
[--vec f32|f16|q8_0] [--no-bm25] [--rebuild] [--graph --gen-model MODEL] [--caption-model MODEL [--caption-mmproj F]]
[--caption-prompt TEXT|@FILE] [--gen-backend cpu|opencl|vulkan|auto] [--progress-json FILE|-]
oklm rag search INDEX QUERY [-k 10] [--mode dense|sparse|hybrid] [--fusion rrf|weighted] [--rrf-k 60] [--weights D,S]
[--rerank MODEL [--rerank-top 50]] [--rag-graph on|off [--graph-hops 1|2]] [--json|--json-full]
oklm rag info INDEX [--sources] [--json]
oklm rag eval DIR --index INDEX [--model MODEL] [--query-prefix TEXT] [--mode dense,sparse,hybrid] [--fusion rrf,weighted]
[--chunk-size N|0] [--depth 100] [--rerank MODEL [--rerank-top 50]] [--run PREFIX] [--json]
A retrieval index of documents in one GGUF file: chunks, their
vectors and BM25 postings (rag.md).
index builds it or updates it in place (unchanged files
kept by their hash), search queries it, info
describes it, eval scores retrieval on a BEIR-format
set.
| Option | Default | Meaning |
|---|---|---|
--out INDEX |
(index) the index written or updated |
|
--model MODEL|hash[:DIM] |
(index, eval) the embedding model, or the
hashed bag of words (rag.md) |
|
--query-prefix TEXT,
--passage-prefix TEXT |
the model's | the text put before a query or a passage |
--chunker auto|fixed|recursive|code|semantic |
auto | how documents are cut |
--chunk-size N, --overlap N |
a chunk's size in tokens, the tokens consecutive fixed windows share | |
--vec f32|f16|q8_0 |
f16 | the vectors' type |
--no-bm25 |
no keyword postings | |
--rebuild |
the existing index ignored | |
--graph --gen-model MODEL,
--gen-backend B |
the entities and relations a generator extracts from each chunk kept (rag.md) | |
--caption-model MODEL, --caption-mmproj F,
--caption-prompt TEXT|@FILE |
the figures of documents indexed as captions by a vision model (rag.md) | |
-k N |
10 | (search) hits printed |
--mode dense|sparse|hybrid |
(search; eval: a list) vectors, keywords,
or both (rag.md) |
|
--fusion rrf|weighted, --rrf-k N,
--weights D,S |
, 60 | how the two rankings are fused |
--rerank MODEL, --rerank-top N |
, 50 | the best N candidates reordered by a reranker |
--rag-graph on|off, --graph-hops 1|2 |
(search) the index's graph walked, and how far |
|
--sources |
(info) each source with its hash and state |
|
--index INDEX, --depth N,
--run PREFIX |
, 100 | (eval) the index scored, how deep the rankings go, the
rankings written in TREC run format |
--json, --json-full,
--progress-json |
rag.md |
Multimodal and speech
oklm describe
oklm describe -m <model> --mmproj <mmproj.gguf> [-p QUESTION] [--tiles SIDE] [-n 256] [--fps 1] [--frames 8] [--ffmpeg] [--chat-template FILE|NAME]
[--vision-id] [--then QUESTION [--then-with FILE]...] [--kv-save FILE] <image, sound or video>...
oklm describe -m <model> --mmproj <mmproj.gguf> --kv-load FILE -p QUESTION [<image, sound or video>...] [--then ...] [--kv-save FILE]
Answers a question about images, sounds or videos with a vision or an
audio model; the projector file decides which (multimodal.md). It takes sampling, -c, --backend
and --chat-template, and:
| Option | Meaning |
|---|---|
--mmproj FILE |
the projector (required) |
-p QUESTION |
the question (default: "Describe this image.", "this video." or "this sound.") |
--tiles SIDE |
a picture's longest side before it is sliced into tiles (0: the encoder's default) |
--fps F, --frames N |
videos: frames sampled a second (1), frames at most (8) (multimodal.md) |
--ffmpeg |
containers other than Ogg Theora decoded through ffmpeg |
--vision-id |
pictures and videos numbered in the prompt, as Qwen2-VL's template does |
--then QUESTION, --then-with FILE |
a second question after the answer, with media of its own |
--kv-save FILE, --kv-load FILE |
the conversation's cache written to a KV file, and taken back in a later run to ask the next question without evaluating the media again |
oklm transcribe
oklm transcribe -m <whisper checkpoint dir> [--language auto|CODE] [--translate] [--timestamps |
--format text|timed|srt|vtt|json] [--no-timestamps] [--detect-language] <audio file>
Speech to text with a Whisper checkpoint (multimodal.md).
| Option | Meaning |
|---|---|
--language auto|CODE |
the language spoken; auto (default): found from the
first 30 seconds |
--translate |
English written whatever the language |
--format text|timed|srt|vtt|json |
the words (default), a line a segment with its times, subtitles, or JSON |
--timestamps |
--format timed |
--no-timestamps |
decoded without timestamp tokens, a cut every 30 seconds |
--detect-language |
the language and its probability printed, nothing transcribed |
--backend B |
where the model computes (default cpu) |
oklm speak
oklm speak -m <speech model> [--vocoder <vocoder.gguf>] [-o speech.wav] [--voice <speaker.json>] [TEXT]
Text to speech into a WAV or FLAC file, with OuteTTS and WavTokenizer's decoder, or Soprano (speech.md). The text is given, or read from standard input.
| Option | Meaning |
|---|---|
--vocoder FILE |
the model that turns the audio codes into sound (found by itself for a catalogue model) |
-o FILE |
the file written (speech.wav; a name ending in
.flac writes FLAC) |
--voice FILE |
a speaker file of OuteTTS in place of the built-in voice |
--temp, --top-k,
--repeat-penalty, --seed |
how the audio codes are sampled |
--no-guide |
the first token of each word left to the model |
--words N, --prompt-words N,
--first-words N |
words a pass (24), words a prompt (64), words of the first pass (speech.md) |
--speed X |
0.25 to 4, the pitch kept |
--codes FILE |
audio codes read from a file, the language model skipped |
--backend B |
where the language model computes (default cpu) |
oklm-gui
oklm-gui holds the graphical tools in one program;
--tool NAME picks one, chat without it (gui.md).
oklm-gui [--tool NAME] [--size WIDTHxHEIGHT] [--screenshot FILE.png]
| Option | Meaning |
|---|---|
--tool NAME |
the tool; --tool list prints the names, one a line |
--size WIDTHxHEIGHT |
the window's size |
--screenshot FILE.png |
the window saved to the file, then the program ends (for checks without a person) |
--help, -h |
the tool's usage |
--tool |
Window | Described in |
|---|---|---|
launcher |
a button for each tool, and the documentation | The launcher |
chat |
a chat with a model the window serves, a server, or a listening agent | The chat window |
models |
the catalogue: download, delete, open a tool on a model | The model manager |
servers |
start, watch and stop oklm serve |
The server manager |
inspect |
a model's layers, where each is and the memory it takes | The model inspector |
convert |
oklm convert and oklm quantize with the
loss measured |
The conversion tool |
train |
oklm train, rl, refusals and
abliterate followed |
The training window |
sessions |
the agent sessions of a project folder | The sessions browser |
rag |
build, check, search and evaluate a retrieval index | The retrieval manager |
oklm-gui --tool launcher
oklm-gui --tool launcher [--press open:TOOL|docs]...
The home window. It takes no option of its own.
oklm-gui --tool chat
oklm-gui [--tool chat] (-m <model.gguf> [-c 4096] [--backend cpu|opencl|vulkan|auto] [--parallel 2]
[--gpu-layers N] [--n-cpu-moe N|all] [--expert-gpu GB|auto] [--kv-mem MB] [-ctk TYPE] [-ctv TYPE] [--devices N]
[--draft MODEL [--draft-n N]] [--spec-ngram N] [--prefix-dir DIR] [--chat-template FILE|NAME] [--tiles SIDE]
[--expert-cache GB|auto [--pin-gb GB] [--expert-pack FILE]]
| --url http(s)://host:port [--key KEY] [--ca FILE] [--no-verify] | --agent PORT)
[--rag INDEX.gguf] [--mmproj FILE] [--attach FILE]... [--ffmpeg] [--embed-model NAME|FILE]
[--rerank-model NAME|FILE | --no-rerank] [--no-doc-cache]
[--chat-file FILE.json]
[--send TEXT]... [--screenshot FILE.png]
[--auto-compact on|off] [--auto-continue on|off] [--continue-max 8]
[--transcribe-model NAME|DIR | --no-transcribe] [--transcribe-language CODE] [--transcribe-translate]
[--transcribe-attached]
[--listen-silence SECONDS] [--listen-threshold DB] [--listen-send] [--no-partial] [--mic NAME] [--mic-file FILE[@TIME]]...
[--talk] [--no-interrupt]
[--speak-model NAME|FILE [--speak-vocoder FILE] [--speak-voice FILE] | --no-speak] [--read-aloud]
[--speak-speed 0.25..4] [--speak-whole]
[--speak-file FILE.wav] [--size 1100x720] [--models] [--model-settings] [--settings 'OPTIONS']...
[--open-servers FILE.png]
[--press speech|models[:NAME]|connect[:PORT]|link|own|choose[:PORT|:URL|:NAME|:file[:PATH]|:mmproj[:PATH]]|use
|advanced[:TAB]|continue|context:N|apply-context|paste|fill:FIELD=VALUE|list-models
|pick-model:NAME|save|forget:URL|model:NAME]...
[--shot-when resizing|recording|speaking|talking|commands:TEXT]
The chat window (Options has each option in full).
| Option | Meaning |
|---|---|
-m MODEL, -c N, --backend B,
--parallel N |
the model the window serves itself on loopback, its context, backend and sequences |
--url URL, --key KEY,
--ca FILE, --no-verify |
a server at an address instead: its API key, a certificate authority file, the certificate not checked |
--agent PORT |
attach to the agent listening on that port (gui.md) |
--models |
open on the Models window |
--model-settings,
--settings 'OPTIONS' |
the model settings open at start; options of oklm serve
set and applied in them, for checks without a person (gui.md) |
--rag INDEX.gguf |
a retrieval index beside the model (gui.md) |
--mmproj FILE, --attach FILE,
--ffmpeg |
the projector, files attached to the first message, videos through ffmpeg (gui.md) |
--embed-model M, --rerank-model M,
--no-rerank, --no-doc-cache |
the models that index and order a document too long for the context; its index not kept |
--chat-file FILE.json |
a saved conversation opened |
--send TEXT |
a message sent at start (repeatable) |
--transcribe-model M, --no-transcribe,
--transcribe-language CODE,
--transcribe-translate,
--transcribe-attached |
speech to text (gui.md) |
--listen-silence S, --listen-threshold DB,
--listen-send, --no-partial,
--mic NAME, --mic-file FILE[@TIME] |
the microphone: the pause that ends a recording, the level that is speech, the words sent at once, no words shown while a recording is still coming, the device, a sound file in its place |
--talk, --no-interrupt |
a spoken conversation; speaking over the voice does not stop it (gui.md) |
--speak-model M, --speak-vocoder FILE,
--speak-voice FILE, --no-speak,
--read-aloud, --speak-speed X,
--speak-whole, --speak-file FILE.wav |
replies read aloud (gui.md) |
oklm-gui --tool models
oklm-gui --tool models [--select NAME] [--installed]
--select NAME selects a model of the catalogue at start;
--installed lists the installed models only.
oklm-gui --tool servers
oklm-gui --tool servers [-m MODEL] [--backend cpu|opencl|vulkan|auto] [-c 4096] [--profile NAME]
[--options 'OPTIONS OF oklm serve'] [--save-profile NAME] [--advanced [TAB]]
[--press start|keep:NAME|profile:NAME|select:T|stop:T|restart:T|unkeep:NAME|rekeep:NAME|
ask-rekeep:NAME|dashboard:T|chat:T|context:N|yes|no]...
-m, --backend and -c are what
the form shows at start; --profile NAME loads a kept
profile in it.
| Option | Meaning |
|---|---|
--options 'OPTIONS' |
a line of options of oklm serve read into the form:
each to its control, the rest to the Advanced window's text field (gui.md) |
--save-profile NAME |
the form kept under that name at start |
--advanced [TAB] |
the Advanced window open at start, on the tab named |
oklm-gui --tool inspect
oklm-gui --tool inspect [-m MODEL.gguf | --url URL [--key KEY]] [--backend cpu|opencl|vulkan] [-c CTX]
[--gpu-layers N] [--n-cpu-moe N] [--ram MIB] [--select embeddings|BLOCK]
[--prompt TEXT [--chat] [--tokens N] [--step N]] [--layer BLOCK] [--view writes|lens|effects]
| Option | Meaning |
|---|---|
-m MODEL.gguf |
a file, read through its tensor table (no weight is loaded) |
--url URL, --key KEY |
a running server, with its real placement |
--backend, -c, --gpu-layers,
--n-cpu-moe |
where the projected placement starts |
--ram MIB |
project against that much host memory |
--select embeddings|BLOCK,
--layer BLOCK |
the part shown as clicked; the block whose attention is shown |
--prompt TEXT, --chat,
--tokens N, --step N |
a prompt run through the model, as a user's message, the tokens generated (8), the token shown |
--view writes|lens|effects |
what the activation panel shows |
oklm-gui --tool convert
oklm-gui --tool convert [-m <model.gguf|hf_dir>] [--out FILE.gguf] [--type Q4_K_M] [--pure] [--imatrix FILE]
[--name NAME] [--text FILE.txt] [-c 512] [--run] [--check]
The options fill the window's fields (-m,
--out, --type, --pure,
--imatrix, --name as the commands';
--text FILE.txt and -c for the perplexity
measure); --run starts the conversion, --check
the measure once the result is there.
oklm-gui --tool train
oklm-gui --tool train [--mode scratch|finetune|lora|chat|rl|advanced] [--set FIELD=VALUE]... [--start]
[--oklm PROGRAM] [--press stop|merge|export|export:FILE|chat]...
--mode is the tab shown; --set FIELD=VALUE
gives a field its value (the usage lists the names);
--start starts the form's command at once;
--oklm PROGRAM is the oklm the commands
run.
oklm-gui --tool sessions
oklm-gui --tool sessions [--project DIR] [--search TEXT] [--session ID]
[--press select:ID|select:live|attach|resume|export|export:FILE|delete|yes|no]...
--project DIR is the project folder,
--search TEXT filters the sessions by a text their messages
hold, --session ID opens one.
oklm-gui --tool rag
oklm-gui --tool rag [--index INDEX.gguf] [--tab build|sources|search|evaluate|attach] [--query TEXT]
[--model NAME|FILE] [--press sources:PATH|build|eval:DIR]...
--index opens an index at start, --tab
chooses the tab, --query searches once the index is read,
--model chooses the embedding model of the Build tab.