Models: the catalogue and its downloads
OKLM ships no model file: not in the repository, not in the
.deb, the AppImage or the Windows installer.
oklm models lists the models it offers to download,
oklm models pull installs them in the user's model
directory, and the name of an installed model can then be given wherever
a model file is expected. This page is the user guide, the catalogue,
how a download is checked, and how the catalogue is maintained.
Status: maintained. Implemented, tested in make test
(tests/client/test_models.c) and run against Hugging Face
on Linux and, for the Windows build, under Wine (2026-11-12).
Using it
oklm models # the catalogue by category, what is installed
oklm models --category code # one category: chat, code, vision, audio, speech, security, retrieval
oklm models pull qwen3-8b # download it (resumed when interrupted)
oklm chat -m qwen3-8b # the name stands for the file
oklm describe -m smolvlm-256m --mmproj smolvlm-256m -p "What is this?" photo.jpg
oklm transcribe -m whisper-base speech.wav
oklm models path qwen3-8b # the file's path, for another program
oklm models rm qwen3-8b # remove it
oklm models dir # where the models are kept
oklm models(oroklm models list) prints, by category, each model's name, size, context, state and purpose. The context column is what the model gets without-c, then the range-ctakes (Context sizes).+marks an installed model,*a file of a supported family that OKLM's authors have not run,!a model larger than this machine's memory, andlicencea model whose licence is asked before the download.--installedkeeps the installed ones,--jsonprints the same as JSON.oklm models pull NAME...downloads every file of each model that is not there yet. A model already installed is left alone.- A name is resolved for
-m,--model,--mmproj,--draft,--embed-model,--rerank-modeland--mtp-model, in every command but those given--url(which name a server's models). A file or a directory of that name in the current directory wins.--mmproj NAMEgives the projector of a vision or audio model. A catalogue name that is not installed is an error that says whichpullto run. oklm-gui -m NAMEtakes the same names. Started without-mand--url(a menu entry, a double click) it loads nothing: it lists the servers running on this machine and the installed chat models, and talks to the one chosen (gui.md). Its Models... button opens the same catalogue in a window, to download, use and delete models (gui.md); with no model installed and no server running the window opens on it.- In
oklm agent,/modellists the installed models with the project's.
The model directory
$OKLM_MODELS_DIR, else
$XDG_DATA_HOME/oklm/models, else
~/.local/share/oklm/models
(%LOCALAPPDATA%\oklm\models on Windows): the directory the
agent already looked in for a model named in its settings. Each model
has a directory of its name holding its files under the names their
repository gives them. licences-accepted.txt there records
the licences agreed to.
Licences
Every pull prints the model's licence and where its text is. A
licence that is more than a permissive one (Meta's Llama licence,
Google's Gemma terms, DeepSeek's model licence, a licence with usage
restrictions, a licence its repository only names "other") has to be
agreed to: the name and the address of the text are shown and the
download starts only after yes is typed. The answer is
recorded once per model, so a later pull of the same model does not ask
again. --accept-licence answers yes without the question;
without a terminal and without that flag, nothing is downloaded.
Context sizes
The context is how many tokens a model works on at once: the conversation, the documents given to it and its answer. Each catalogue entry carries the context its model was trained with, read from the file's header when the catalogue is generated.
- Default. Without
-c,run,chat,serveandoklm-guigive a model 4096 tokens, or its trained context when that is smaller;oklm agentasks for 32768, capped at the trained context. - Range.
-c Ntakes any size; the list gives 256 (below it there is little room for an answer) to the trained context, where the model stops having been taught positions. - Above the trained context. Allowed everywhere, with
a warning in red:
run,chatandserveprint it at start, the window shows it under its slider (which goes to 1 048 576 tokens whatever the model), the agent prints it and takes up to 1 048 576 too, and a server says it in its answer. A model whose positions are a stored table (GPT-2, the embedding models) is capped at that table whatever is asked. - Memory. The key and value cache grows with the
context:
oklm plan -m NAME -c Nsays what a size needs. A size the plan says does not fit is warned of in red: at start byrun,chatandserve, which go on; by the agent, which gives the figures for the size asked (memory in all, the cache's part, the device's) and keeps the old size unlessforceis added; by a server whose cache is in host memory, with507unless"force": true. The window's line under its slider always gives the memory of the size shown, as the slider moves, and turns red when it does not fit (gui.md).oklm modelsprints in red the models larger than the machine's memory. - Fixed: Whisper. Its window is not a setting. The
encoder reads 30 seconds of sound as 1500 positions and the decoder
writes at most 448 tokens for them, and both numbers are the sizes of
position tables that are part of the trained weights: a larger one has
no rows to read, and a smaller one would only cut sound or text off.
Longer recordings are transcribed a window of 30 seconds at a time, each
starting where the last whole segment of the one before ended, which
oklm transcribedoes.
Changing it while running:
oklm-gui: the side panel's Context slider and Apply context (gui.md).oklm agent:/context-size N(alias/ctx);/context-sizealone prints the size and the range (agent-ui.md).oklm serve:POST /api/context(server.md), which the window and the agent use when their model is on a server;--fixed-contextturns it off.oklm chat,oklm run: at start only (-c).
In both places a change makes a new context for the same loaded model: the weights stay in memory, the conversation is kept, and the model reads it again on the next message, since the cache of the old context cannot be carried into one of another size.
The catalogue
Sizes are the files' on disk. "Run here": the very file the catalogue
names was loaded and generated text (or transcribed, embedded,
described) on the authors' machine; "no" is a file of a family OKLM
supports (compatibility.md) that could
not be run there, which oklm info and
oklm plan check once it is downloaded.
| Category | Name | Model | Size | Context: default (range of -c) |
Licence | Run here |
|---|---|---|---|---|---|---|
| General chat | smollm2-135m |
SmolLM2-135M-Instruct Q8_0 | 138 MiB | 4096 (256 to 8192) | apache-2.0 | yes |
| General chat | qwen3-0.6b |
Qwen3-0.6B Q8_0 | 610 MiB | 4096 (256 to 40960) | apache-2.0 | yes |
| General chat | llama3.2-1b |
Llama-3.2-1B-Instruct Q4_K_M | 770 MiB | 4096 (256 to 131072) | llama3.2 (asked) | yes |
| General chat | gemma3-1b |
Gemma-3-1B-it Q4_K_M | 769 MiB | 4096 (256 to 32768) | gemma (asked) | yes |
| General chat | phi3.5-mini |
Phi-3.5-mini-instruct Q4_K_M | 2.2 GiB | 4096 (256 to 131072) | mit | yes |
| General chat | qwen3-4b |
Qwen3-4B Q4_K_M | 2.3 GiB | 4096 (256 to 40960) | apache-2.0 | yes |
| General chat | qwen3-8b |
Qwen3-8B Q4_K_M | 4.7 GiB | 4096 (256 to 40960) | apache-2.0 | yes |
| General chat | qwen3.5-9b |
Qwen3.5-9B Q4_K_M | 5.3 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| General chat | qwen3.8-27b |
Qwen3.8-27B Q4_K_M | 15.4 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| General chat | ternary-bonsai-2-2b |
Ternary Bonsai 2 2B PQ2_0 | 493 MiB | 4096 (256 to 262144) | apache-2.0 | yes |
| General chat | ternary-bonsai-2-4b |
Ternary Bonsai 2 4B PQ2_0 | 1.1 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| General chat | ternary-bonsai-2-9b |
Ternary Bonsai 2 9B PQ2_0 | 2.3 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| General chat | glm4-9b |
GLM-4-9B-0414 Q4_K_M | 5.7 GiB | 4096 (256 to 32768) | mit | yes |
| General chat | glm-z1-9b |
GLM-Z1-9B-0414 Q4_K_M | 5.7 GiB | 4096 (256 to 32768) | mit | yes |
| General chat | phi4-14b |
Phi-4 (14B) Q4_K_M | 8.4 GiB | 4096 (256 to 16384) | mit | no |
| General chat | precog-24b |
Precog-24B-v1 Q4_K_M | 13.3 GiB | 4096 (256 to 131072) | not stated by its author (base model: apache-2.0) (asked) | no |
| General chat | glm4-32b |
GLM-4-32B-0414 Q4_K_M | 18.3 GiB | 4096 (256 to 32768) | mit | yes |
| General chat | glm-4.7-flash |
GLM-4.7-Flash Q4_K_M | 17.1 GiB | 4096 (256 to 202752) | mit | yes |
| General chat | glm-4.5-air |
GLM-4.5-Air Q4_K_M | 68.0 GiB | 4096 (256 to 131072) | mit | no |
| General chat | gpt-oss-20b |
gpt-oss-20b Q4_K_M | 10.8 GiB | 4096 (256 to 131072) | apache-2.0 | yes |
| General chat | spark-x2.6-4b |
Spark-X2.6 4B Q4_K_M | 3.3 GiB | 4096 (256 to 1048576) | apache-2.0 | yes |
| General chat | spark-x2.6-1.7b |
Spark-X2.6 1.7B Q4_K_M | 1.6 GiB | 4096 (256 to 1048576) | apache-2.0 | yes |
| General chat | muse-glimmer-30b |
Muse Glimmer 30B UD-IQ2_XXS | 10.0 GiB | 4096 (256 to 131072) | apache-2.0 | yes |
| General chat | gemma4-12b |
Gemma-4-12B-it Q4_K_M | 6.6 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| General chat | gemma4-26b-a4b |
Gemma-4-26B-A4B-it Q4_K_M | 15.8 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| General chat | gemma4-e2b |
Gemma-4-E2B-it Q4_K_M | 2.9 GiB | 4096 (256 to 131072) | apache-2.0 | yes |
| General chat | gemma4-e4b |
Gemma-4-E4B-it Q4_K_M | 4.6 GiB | 4096 (256 to 131072) | apache-2.0 | yes |
| Coding | qwen3-8b |
Qwen3-8B Q4_K_M | 4.7 GiB | 4096 (256 to 40960) | apache-2.0 | yes |
| Coding | glm4-32b |
GLM-4-32B-0414 Q4_K_M | 18.3 GiB | 4096 (256 to 32768) | mit | yes |
| Coding | glm-4.7-flash |
GLM-4.7-Flash Q4_K_M | 17.1 GiB | 4096 (256 to 202752) | mit | yes |
| Coding | glm-4.5-air |
GLM-4.5-Air Q4_K_M | 68.0 GiB | 4096 (256 to 131072) | mit | no |
| Coding | qwen2.5-coder-1.5b |
Qwen2.5-Coder-1.5B-Instruct Q4_K_M | 1.0 GiB | 4096 (256 to 32768) | apache-2.0 | yes |
| Coding | starcoder2-3b |
StarCoder2-3B Q4_K_M | 1.7 GiB | 4096 (256 to 16384) | bigcode-openrail-m (asked) | yes |
| Coding | deepseek-coder-1.3b |
DeepSeek-Coder-1.3B-Instruct Q4_K_M | 833 MiB | 4096 (256 to 16384) | deepseek (use restrictions) (asked) | yes |
| Coding | deepseek-coder-6.7b |
DeepSeek-Coder-6.7B-Instruct Q4_K_M | 3.8 GiB | 4096 (256 to 16384) | deepseek (use restrictions) (asked) | yes |
| Coding | qwen2.5-coder-7b |
Qwen2.5-Coder-7B-Instruct Q4_K_M | 4.4 GiB | 4096 (256 to 131072) | apache-2.0 | yes |
| Coding | qwen2.5-coder-14b |
Qwen2.5-Coder-14B-Instruct Q4_K_M | 8.4 GiB | 4096 (256 to 131072) | apache-2.0 | yes |
| Coding | codestral-22b |
Codestral-22B-v0.1 Q4_K_M | 12.4 GiB | 4096 (256 to 32768) | MNPL-0.1 (Mistral non-production licence) (asked) | no |
| Coding | devstral-small-2-24b |
Devstral Small 2 24B Instruct (2512) Q4_K_M | 13.3 GiB | 4096 (256 to 393216) | apache-2.0 | yes |
| Coding | qwen3-coder-30b |
Qwen3-Coder-30B-A3B-Instruct Q4_K_M | 17.3 GiB | 4096 (256 to 262144) | apache-2.0 | no |
| Coding | mimo-v2.6-distill-qwen-9b |
MiMo-V2.6-Distill-Qwen-9B Q4_K_M | 5.4 GiB | 4096 (256 to 262144) | mit | yes |
| Images | smolvlm-256m |
SmolVLM-256M-Instruct Q8_0 | 348 MiB | 4096 (256 to 8192) | apache-2.0 | yes |
| Images | smolvlm-500m |
SmolVLM-500M-Instruct Q8_0 | 607 MiB | 4096 (256 to 8192) | apache-2.0 | yes |
| Images | smolvlm2-2.2b |
SmolVLM2-2.2B-Instruct Q8_0 | 2.6 GiB | 4096 (256 to 8192) | apache-2.0 | yes |
| Images | gemma3-4b |
Gemma-3-4B-it Q4_K_M | 3.1 GiB | 4096 (256 to 131072) | gemma (asked) | yes |
| Images | llava-1.5-7b |
LLaVA-1.5-7B Q4_K | 4.4 GiB | 4096 (256 to 4096) | llama2 (asked) | yes |
| Images | llava-1.6-mistral-7b |
LLaVA-1.6-Mistral-7B Q4_K_M | 4.6 GiB | 4096 (256 to 32768) | apache-2.0 | yes |
| Images | qwen2-vl-2b |
Qwen2-VL-2B-Instruct Q4_K_M | 2.2 GiB | 4096 (256 to 32768) | apache-2.0 | yes |
| Images | qwen2.5-vl-7b |
Qwen2.5-VL-7B-Instruct Q4_K_M | 5.6 GiB | 4096 (256 to 128000) | apache-2.0 | yes |
| Images | qwen2.5-vl-3b |
Qwen2.5-VL-3B-Instruct Q4_K_M | 3.0 GiB | 4096 (256 to 128000) | qwen-research (asked) | yes |
| Images | qwen2.5-vl-32b |
Qwen2.5-VL-32B-Instruct Q4_K_M | 19.8 GiB | 4096 (256 to 128000) | apache-2.0 | yes |
| Images | qwen2.5-vl-72b |
Qwen2.5-VL-72B-Instruct IQ1_M | 23.4 GiB | 4096 (256 to 128000) | qwen (asked) | yes |
| Sounds | whisper-base |
Whisper base | 281 MiB | 448, fixed | apache-2.0 | yes |
| Sounds | whisper-small |
Whisper small | 926 MiB | 448, fixed | apache-2.0 | yes |
| Sounds | whisper-large-v3-turbo |
Whisper large-v3-turbo | 1.5 GiB | 448, fixed | mit | yes |
| Sounds | whisper-large-v3 |
Whisper large-v3 | 2.9 GiB | 448, fixed | apache-2.0 | yes |
| Sounds | ultravox-1b |
Ultravox v0.5 (Llama-3.2-1B) Q8_0 | 2.5 GiB | 4096 (256 to 131072) | mit, llama3.2 (asked) | yes |
| Speech synthesis | outetts-0.2-500m |
OuteTTS 0.2 500M Q8_0 with WavTokenizer large (F16) | 636 MiB | 4096, fixed | cc-by-nc-4.0 (the vocoder: mit) (asked) | yes |
| Speech synthesis | outetts-0.3-500m |
OuteTTS 0.3 500M Q8_0 with WavTokenizer large (F16) | 636 MiB | 4096, fixed | cc-by-sa-4.0 (the vocoder: mit) | yes |
| Speech synthesis | soprano-1.1-80m |
Soprano 1.1 80M Q8_0 with its decoder (F32) | 197 MiB | 1024, fixed | apache-2.0 | yes |
| Security | cybersec-assistant-3b |
CyberSec-Assistant-3B Q4_K_M | 1.8 GiB | 4096 (256 to 32768) | apache-2.0 | yes |
| Security | foundation-sec-8b-instruct |
Foundation-Sec-1.1-8B-Instruct Q4_K_M | 4.6 GiB | 4096 (256 to 65536) | other (see the model's page) (asked) | yes |
| Security | foundation-sec-8b |
Foundation-Sec-8B Q4_K_M | 4.6 GiB | 4096 (256 to 131072) | apache-2.0 | yes |
| Security | whiterabbitneo-2.5-7b |
WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B Q4_K_M | 4.4 GiB | 4096 (256 to 32768) | apache-2.0 with usage restrictions (asked) | yes |
| Security | cyber-ornith-1.5-9b |
Cyber-Ornith 1.5 9B Q4_K_M | 5.2 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| Security | bugtraceai-core-ultra-27b |
BugTraceAI-CORE-Ultra 27B Q4_K_S | 14.5 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| Security | uraion-cyber-v1.0 |
Uraion Cyber V1.0 Q4_K_M | 19.7 GiB | 4096 (256 to 262144) | mit | yes |
| Security | imperum-cybersecurityllm-v1.0 |
Imperum-CybersecurityLLM v1.0 Q4_K_M | 19.7 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| Security | cyberstrike-offsec-35b |
CyberStrike-OffSec-35B Q4_K_M | 20.2 GiB | 4096 (256 to 262144) | apache-2.0 | yes |
| Security | cyber-prime-1.1-2.6b |
Cyber-Prime 1.1 2.6B Q4_K_M | 1.6 GiB | 4096 (256 to 131072) | other: lfm1.0 (asked) | yes |
| Security | tini-cybersec-8b-a1b |
Tini-Cybersec 8B-A1B Q4_K_M | 4.8 GiB | 4096 (256 to 128000) | apache-2.0 (base model: lfm1.0) (asked) | yes |
| Retrieval | bge-small-en-v1.5 |
bge-small-en-v1.5 F32 | 127 MiB | 512 (256 to 512) | mit | yes |
| Retrieval | nomic-embed-text-v1.5 |
nomic-embed-text-v1.5 Q8_0 | 139 MiB | 2048 (256 to 2048) | apache-2.0 | yes |
| Retrieval | qwen3-embedding-0.6b |
Qwen3-Embedding-0.6B Q8_0 | 610 MiB | 4096 (256 to 32768) | apache-2.0 | yes |
| Retrieval | ms-marco-minilm-l6-v2 |
ms-marco-MiniLM-L6-v2 F16 | 44 MiB | 512 (256 to 512) | apache-2.0 | yes |
What the catalogue cannot offer yet, and what each needs, is on the
roadmap (larger work, L1, L5
and L7): vision models other than SmolVLM, Gemma 3, LLaVA 1.5 and 1.6,
Qwen2-VL and Qwen2.5-VL (GLM-4.5V: its projector is refused; Muse
Glimmer is offered for text, its perception encoder is not read) and
Llama 3.2 Vision. GLM-4.5-Air, Phi-4, Precog-24B, Codestral-22B and
Qwen3-Coder-30B are listed without having been run:
oklm info finds their headers loadable and their files did
not fit on the authors' disk. Precog-24B, a story and role-play tune, is
under general chat: the catalogue has no category of its own for it.
How a download is checked
- Pinned. Each file is named by a URL at one revision
of its repository (a commit, not a branch), with its size and SHA-256
compiled into
oklm: what is downloaded tomorrow is what was listed. - Resumed. The bytes go to
FILE.part; an interrupted pull asks for the rest with aRangerequest, and a connection cut in the middle is taken up again as long as bytes keep arriving. A server that answers the whole file instead starts the file again; a partial file larger than the file is dropped. - Verified. The file is renamed into place only when its size and its SHA-256 are the catalogue's; otherwise it is removed and the pull fails.
- TLS. The certificate chain and the host name are
checked against the system's authorities (on Windows, the system's root
store). A build without OpenSSL cannot download from
httpsand says so; the Windows build has OpenSSL oncemake windows-depsbuilt it (build.md). - Redirects are followed (eight at most), to another host as well: Hugging Face answers a file with a redirect to its storage.
oklm_models_pull(), oklm_models_download()
and the catalogue (oklm_catalog_get(),
oklm_catalog_find()) are in
include/oklm/client/models.h;
oklm_client_get_data() in
include/oklm/client/client.h is the streamed GET under
them.
Maintaining the catalogue
tools/model_catalog.json is the list: a name, the
categories, a title, one line of purpose, the licence, whether it is
asked, whether the file was run, and each file's repository and path. A
model whose file carries no chat template names its conversation format
there ("chat_template": one of the names
--chat-template takes, inference.md), read
from the model's card and never guessed: vicuna for LLaVA
1.5 (its card: tuned from Vicuna, the prompt
USER: ... ASSISTANT:), deepseek-coder for the
two DeepSeek-Coder files (their tokenizer_config.json),
mistral-v3 for Codestral 22B (Mistral's
tokenizer_config.json; the quantizer's card prints another
form) and foundation-sec for Foundation-Sec-1.1-8B-Instruct
(its chat_template.jinja). The commands then chat with the
model in that format without an option; StarCoder2 and Foundation-Sec-8B
are completion models and name none.
python3 tools/gen_model_catalog.py --local models pins each
repository's current revision, reads the sizes and SHA-256 sums from the
repository's listing, writes src/client/model_catalog.inc
(committed) and compares every listed file it finds under
models/ with its sum.
python3 tools/docs/models_table.py --write then rewrites
the table above. A model is marked as run only when the file the
catalogue names was run: for the entries added on 2026-11-12 each was
pulled with oklm models pull, run on the CPU and removed.
Qwen3.8-27B is marked from the record of its run of 2026-11-13
(tests/model/test_infer_public.c against llama.cpp: the
file of that SHA-256, removed since for lack of disk). Cyber-Ornith 1.5
9B and BugTraceAI-CORE-Ultra 27B were run the same day against llama.cpp
and in oklm chat, then removed (compatibility.md, the qwen35
row, has the figures), and so were Muse Glimmer 30B and the three tunes
of Qwen3.6-35B-A3B (the muse-glimmer and
qwen35moe rows), one file on the disk at a time. Qwen2.5-VL
3B, 32B and 72B were fetched, run against llama.cpp and on both devices,
and removed on 2026-11-14 (multimodal.md);
their three entries were added to the generated table by hand in the
generator's form (the revisions, sizes and sums the repositories listed
that day, the sums those of the files run), so that the other entries'
revisions did not move. The 72B's file is its IQ1_M, the one that was
run: a disk with room takes a larger quantization of the same repository
by its file name. MiMo-V2.6-Distill-Qwen-9B was fetched, run against
llama.cpp and in oklm chat on the CPU, and removed on
2026-11-15 (compatibility.md, the
qwen35 row); the generator ran whole that day and moved no
other entry.