OKLMdocumentation v1.0.3
On this page

Models: the catalogue and its downloads

OKLM ships no model file: not in the repository, not in the .deb, the AppImage or the Windows installer. oklm models lists the models it offers to download, oklm models pull installs them in the user's model directory, and the name of an installed model can then be given wherever a model file is expected. This page is the user guide, the catalogue, how a download is checked, and how the catalogue is maintained.

Status: maintained. Implemented, tested in make test (tests/client/test_models.c) and run against Hugging Face on Linux and, for the Windows build, under Wine (2026-11-12).

Using it

oklm models                          # the catalogue by category, what is installed
oklm models --category code          # one category: chat, code, vision, audio, speech, security, retrieval
oklm models pull qwen3-8b            # download it (resumed when interrupted)
oklm chat -m qwen3-8b                # the name stands for the file
oklm describe -m smolvlm-256m --mmproj smolvlm-256m -p "What is this?" photo.jpg
oklm transcribe -m whisper-base speech.wav
oklm models path qwen3-8b            # the file's path, for another program
oklm models rm qwen3-8b              # remove it
oklm models dir                      # where the models are kept
  • oklm models (or oklm models list) prints, by category, each model's name, size, context, state and purpose. The context column is what the model gets without -c, then the range -c takes (Context sizes). + marks an installed model, * a file of a supported family that OKLM's authors have not run, ! a model larger than this machine's memory, and licence a model whose licence is asked before the download. --installed keeps the installed ones, --json prints the same as JSON.
  • oklm models pull NAME... downloads every file of each model that is not there yet. A model already installed is left alone.
  • A name is resolved for -m, --model, --mmproj, --draft, --embed-model, --rerank-model and --mtp-model, in every command but those given --url (which name a server's models). A file or a directory of that name in the current directory wins. --mmproj NAME gives the projector of a vision or audio model. A catalogue name that is not installed is an error that says which pull to run.
  • oklm-gui -m NAME takes the same names. Started without -m and --url (a menu entry, a double click) it loads nothing: it lists the servers running on this machine and the installed chat models, and talks to the one chosen (gui.md). Its Models... button opens the same catalogue in a window, to download, use and delete models (gui.md); with no model installed and no server running the window opens on it.
  • In oklm agent, /model lists the installed models with the project's.

The model directory

$OKLM_MODELS_DIR, else $XDG_DATA_HOME/oklm/models, else ~/.local/share/oklm/models (%LOCALAPPDATA%\oklm\models on Windows): the directory the agent already looked in for a model named in its settings. Each model has a directory of its name holding its files under the names their repository gives them. licences-accepted.txt there records the licences agreed to.

Licences

Every pull prints the model's licence and where its text is. A licence that is more than a permissive one (Meta's Llama licence, Google's Gemma terms, DeepSeek's model licence, a licence with usage restrictions, a licence its repository only names "other") has to be agreed to: the name and the address of the text are shown and the download starts only after yes is typed. The answer is recorded once per model, so a later pull of the same model does not ask again. --accept-licence answers yes without the question; without a terminal and without that flag, nothing is downloaded.

Context sizes

The context is how many tokens a model works on at once: the conversation, the documents given to it and its answer. Each catalogue entry carries the context its model was trained with, read from the file's header when the catalogue is generated.

  • Default. Without -c, run, chat, serve and oklm-gui give a model 4096 tokens, or its trained context when that is smaller; oklm agent asks for 32768, capped at the trained context.
  • Range. -c N takes any size; the list gives 256 (below it there is little room for an answer) to the trained context, where the model stops having been taught positions.
  • Above the trained context. Allowed everywhere, with a warning in red: run, chat and serve print it at start, the window shows it under its slider (which goes to 1 048 576 tokens whatever the model), the agent prints it and takes up to 1 048 576 too, and a server says it in its answer. A model whose positions are a stored table (GPT-2, the embedding models) is capped at that table whatever is asked.
  • Memory. The key and value cache grows with the context: oklm plan -m NAME -c N says what a size needs. A size the plan says does not fit is warned of in red: at start by run, chat and serve, which go on; by the agent, which gives the figures for the size asked (memory in all, the cache's part, the device's) and keeps the old size unless force is added; by a server whose cache is in host memory, with 507 unless "force": true. The window's line under its slider always gives the memory of the size shown, as the slider moves, and turns red when it does not fit (gui.md). oklm models prints in red the models larger than the machine's memory.
  • Fixed: Whisper. Its window is not a setting. The encoder reads 30 seconds of sound as 1500 positions and the decoder writes at most 448 tokens for them, and both numbers are the sizes of position tables that are part of the trained weights: a larger one has no rows to read, and a smaller one would only cut sound or text off. Longer recordings are transcribed a window of 30 seconds at a time, each starting where the last whole segment of the one before ended, which oklm transcribe does.

Changing it while running:

  • oklm-gui: the side panel's Context slider and Apply context (gui.md).
  • oklm agent: /context-size N (alias /ctx); /context-size alone prints the size and the range (agent-ui.md).
  • oklm serve: POST /api/context (server.md), which the window and the agent use when their model is on a server; --fixed-context turns it off.
  • oklm chat, oklm run: at start only (-c).

In both places a change makes a new context for the same loaded model: the weights stay in memory, the conversation is kept, and the model reads it again on the next message, since the cache of the old context cannot be carried into one of another size.

The catalogue

Sizes are the files' on disk. "Run here": the very file the catalogue names was loaded and generated text (or transcribed, embedded, described) on the authors' machine; "no" is a file of a family OKLM supports (compatibility.md) that could not be run there, which oklm info and oklm plan check once it is downloaded.

Category Name Model Size Context: default (range of -c) Licence Run here
General chat smollm2-135m SmolLM2-135M-Instruct Q8_0 138 MiB 4096 (256 to 8192) apache-2.0 yes
General chat qwen3-0.6b Qwen3-0.6B Q8_0 610 MiB 4096 (256 to 40960) apache-2.0 yes
General chat llama3.2-1b Llama-3.2-1B-Instruct Q4_K_M 770 MiB 4096 (256 to 131072) llama3.2 (asked) yes
General chat gemma3-1b Gemma-3-1B-it Q4_K_M 769 MiB 4096 (256 to 32768) gemma (asked) yes
General chat phi3.5-mini Phi-3.5-mini-instruct Q4_K_M 2.2 GiB 4096 (256 to 131072) mit yes
General chat qwen3-4b Qwen3-4B Q4_K_M 2.3 GiB 4096 (256 to 40960) apache-2.0 yes
General chat qwen3-8b Qwen3-8B Q4_K_M 4.7 GiB 4096 (256 to 40960) apache-2.0 yes
General chat qwen3.5-9b Qwen3.5-9B Q4_K_M 5.3 GiB 4096 (256 to 262144) apache-2.0 yes
General chat qwen3.8-27b Qwen3.8-27B Q4_K_M 15.4 GiB 4096 (256 to 262144) apache-2.0 yes
General chat ternary-bonsai-2-2b Ternary Bonsai 2 2B PQ2_0 493 MiB 4096 (256 to 262144) apache-2.0 yes
General chat ternary-bonsai-2-4b Ternary Bonsai 2 4B PQ2_0 1.1 GiB 4096 (256 to 262144) apache-2.0 yes
General chat ternary-bonsai-2-9b Ternary Bonsai 2 9B PQ2_0 2.3 GiB 4096 (256 to 262144) apache-2.0 yes
General chat glm4-9b GLM-4-9B-0414 Q4_K_M 5.7 GiB 4096 (256 to 32768) mit yes
General chat glm-z1-9b GLM-Z1-9B-0414 Q4_K_M 5.7 GiB 4096 (256 to 32768) mit yes
General chat phi4-14b Phi-4 (14B) Q4_K_M 8.4 GiB 4096 (256 to 16384) mit no
General chat precog-24b Precog-24B-v1 Q4_K_M 13.3 GiB 4096 (256 to 131072) not stated by its author (base model: apache-2.0) (asked) no
General chat glm4-32b GLM-4-32B-0414 Q4_K_M 18.3 GiB 4096 (256 to 32768) mit yes
General chat glm-4.7-flash GLM-4.7-Flash Q4_K_M 17.1 GiB 4096 (256 to 202752) mit yes
General chat glm-4.5-air GLM-4.5-Air Q4_K_M 68.0 GiB 4096 (256 to 131072) mit no
General chat gpt-oss-20b gpt-oss-20b Q4_K_M 10.8 GiB 4096 (256 to 131072) apache-2.0 yes
General chat spark-x2.6-4b Spark-X2.6 4B Q4_K_M 3.3 GiB 4096 (256 to 1048576) apache-2.0 yes
General chat spark-x2.6-1.7b Spark-X2.6 1.7B Q4_K_M 1.6 GiB 4096 (256 to 1048576) apache-2.0 yes
General chat muse-glimmer-30b Muse Glimmer 30B UD-IQ2_XXS 10.0 GiB 4096 (256 to 131072) apache-2.0 yes
General chat gemma4-12b Gemma-4-12B-it Q4_K_M 6.6 GiB 4096 (256 to 262144) apache-2.0 yes
General chat gemma4-26b-a4b Gemma-4-26B-A4B-it Q4_K_M 15.8 GiB 4096 (256 to 262144) apache-2.0 yes
General chat gemma4-e2b Gemma-4-E2B-it Q4_K_M 2.9 GiB 4096 (256 to 131072) apache-2.0 yes
General chat gemma4-e4b Gemma-4-E4B-it Q4_K_M 4.6 GiB 4096 (256 to 131072) apache-2.0 yes
Coding qwen3-8b Qwen3-8B Q4_K_M 4.7 GiB 4096 (256 to 40960) apache-2.0 yes
Coding glm4-32b GLM-4-32B-0414 Q4_K_M 18.3 GiB 4096 (256 to 32768) mit yes
Coding glm-4.7-flash GLM-4.7-Flash Q4_K_M 17.1 GiB 4096 (256 to 202752) mit yes
Coding glm-4.5-air GLM-4.5-Air Q4_K_M 68.0 GiB 4096 (256 to 131072) mit no
Coding qwen2.5-coder-1.5b Qwen2.5-Coder-1.5B-Instruct Q4_K_M 1.0 GiB 4096 (256 to 32768) apache-2.0 yes
Coding starcoder2-3b StarCoder2-3B Q4_K_M 1.7 GiB 4096 (256 to 16384) bigcode-openrail-m (asked) yes
Coding deepseek-coder-1.3b DeepSeek-Coder-1.3B-Instruct Q4_K_M 833 MiB 4096 (256 to 16384) deepseek (use restrictions) (asked) yes
Coding deepseek-coder-6.7b DeepSeek-Coder-6.7B-Instruct Q4_K_M 3.8 GiB 4096 (256 to 16384) deepseek (use restrictions) (asked) yes
Coding qwen2.5-coder-7b Qwen2.5-Coder-7B-Instruct Q4_K_M 4.4 GiB 4096 (256 to 131072) apache-2.0 yes
Coding qwen2.5-coder-14b Qwen2.5-Coder-14B-Instruct Q4_K_M 8.4 GiB 4096 (256 to 131072) apache-2.0 yes
Coding codestral-22b Codestral-22B-v0.1 Q4_K_M 12.4 GiB 4096 (256 to 32768) MNPL-0.1 (Mistral non-production licence) (asked) no
Coding devstral-small-2-24b Devstral Small 2 24B Instruct (2512) Q4_K_M 13.3 GiB 4096 (256 to 393216) apache-2.0 yes
Coding qwen3-coder-30b Qwen3-Coder-30B-A3B-Instruct Q4_K_M 17.3 GiB 4096 (256 to 262144) apache-2.0 no
Coding mimo-v2.6-distill-qwen-9b MiMo-V2.6-Distill-Qwen-9B Q4_K_M 5.4 GiB 4096 (256 to 262144) mit yes
Images smolvlm-256m SmolVLM-256M-Instruct Q8_0 348 MiB 4096 (256 to 8192) apache-2.0 yes
Images smolvlm-500m SmolVLM-500M-Instruct Q8_0 607 MiB 4096 (256 to 8192) apache-2.0 yes
Images smolvlm2-2.2b SmolVLM2-2.2B-Instruct Q8_0 2.6 GiB 4096 (256 to 8192) apache-2.0 yes
Images gemma3-4b Gemma-3-4B-it Q4_K_M 3.1 GiB 4096 (256 to 131072) gemma (asked) yes
Images llava-1.5-7b LLaVA-1.5-7B Q4_K 4.4 GiB 4096 (256 to 4096) llama2 (asked) yes
Images llava-1.6-mistral-7b LLaVA-1.6-Mistral-7B Q4_K_M 4.6 GiB 4096 (256 to 32768) apache-2.0 yes
Images qwen2-vl-2b Qwen2-VL-2B-Instruct Q4_K_M 2.2 GiB 4096 (256 to 32768) apache-2.0 yes
Images qwen2.5-vl-7b Qwen2.5-VL-7B-Instruct Q4_K_M 5.6 GiB 4096 (256 to 128000) apache-2.0 yes
Images qwen2.5-vl-3b Qwen2.5-VL-3B-Instruct Q4_K_M 3.0 GiB 4096 (256 to 128000) qwen-research (asked) yes
Images qwen2.5-vl-32b Qwen2.5-VL-32B-Instruct Q4_K_M 19.8 GiB 4096 (256 to 128000) apache-2.0 yes
Images qwen2.5-vl-72b Qwen2.5-VL-72B-Instruct IQ1_M 23.4 GiB 4096 (256 to 128000) qwen (asked) yes
Sounds whisper-base Whisper base 281 MiB 448, fixed apache-2.0 yes
Sounds whisper-small Whisper small 926 MiB 448, fixed apache-2.0 yes
Sounds whisper-large-v3-turbo Whisper large-v3-turbo 1.5 GiB 448, fixed mit yes
Sounds whisper-large-v3 Whisper large-v3 2.9 GiB 448, fixed apache-2.0 yes
Sounds ultravox-1b Ultravox v0.5 (Llama-3.2-1B) Q8_0 2.5 GiB 4096 (256 to 131072) mit, llama3.2 (asked) yes
Speech synthesis outetts-0.2-500m OuteTTS 0.2 500M Q8_0 with WavTokenizer large (F16) 636 MiB 4096, fixed cc-by-nc-4.0 (the vocoder: mit) (asked) yes
Speech synthesis outetts-0.3-500m OuteTTS 0.3 500M Q8_0 with WavTokenizer large (F16) 636 MiB 4096, fixed cc-by-sa-4.0 (the vocoder: mit) yes
Speech synthesis soprano-1.1-80m Soprano 1.1 80M Q8_0 with its decoder (F32) 197 MiB 1024, fixed apache-2.0 yes
Security cybersec-assistant-3b CyberSec-Assistant-3B Q4_K_M 1.8 GiB 4096 (256 to 32768) apache-2.0 yes
Security foundation-sec-8b-instruct Foundation-Sec-1.1-8B-Instruct Q4_K_M 4.6 GiB 4096 (256 to 65536) other (see the model's page) (asked) yes
Security foundation-sec-8b Foundation-Sec-8B Q4_K_M 4.6 GiB 4096 (256 to 131072) apache-2.0 yes
Security whiterabbitneo-2.5-7b WhiteRabbitNeo-2.5-Qwen-2.5-Coder-7B Q4_K_M 4.4 GiB 4096 (256 to 32768) apache-2.0 with usage restrictions (asked) yes
Security cyber-ornith-1.5-9b Cyber-Ornith 1.5 9B Q4_K_M 5.2 GiB 4096 (256 to 262144) apache-2.0 yes
Security bugtraceai-core-ultra-27b BugTraceAI-CORE-Ultra 27B Q4_K_S 14.5 GiB 4096 (256 to 262144) apache-2.0 yes
Security uraion-cyber-v1.0 Uraion Cyber V1.0 Q4_K_M 19.7 GiB 4096 (256 to 262144) mit yes
Security imperum-cybersecurityllm-v1.0 Imperum-CybersecurityLLM v1.0 Q4_K_M 19.7 GiB 4096 (256 to 262144) apache-2.0 yes
Security cyberstrike-offsec-35b CyberStrike-OffSec-35B Q4_K_M 20.2 GiB 4096 (256 to 262144) apache-2.0 yes
Security cyber-prime-1.1-2.6b Cyber-Prime 1.1 2.6B Q4_K_M 1.6 GiB 4096 (256 to 131072) other: lfm1.0 (asked) yes
Security tini-cybersec-8b-a1b Tini-Cybersec 8B-A1B Q4_K_M 4.8 GiB 4096 (256 to 128000) apache-2.0 (base model: lfm1.0) (asked) yes
Retrieval bge-small-en-v1.5 bge-small-en-v1.5 F32 127 MiB 512 (256 to 512) mit yes
Retrieval nomic-embed-text-v1.5 nomic-embed-text-v1.5 Q8_0 139 MiB 2048 (256 to 2048) apache-2.0 yes
Retrieval qwen3-embedding-0.6b Qwen3-Embedding-0.6B Q8_0 610 MiB 4096 (256 to 32768) apache-2.0 yes
Retrieval ms-marco-minilm-l6-v2 ms-marco-MiniLM-L6-v2 F16 44 MiB 512 (256 to 512) apache-2.0 yes

What the catalogue cannot offer yet, and what each needs, is on the roadmap (larger work, L1, L5 and L7): vision models other than SmolVLM, Gemma 3, LLaVA 1.5 and 1.6, Qwen2-VL and Qwen2.5-VL (GLM-4.5V: its projector is refused; Muse Glimmer is offered for text, its perception encoder is not read) and Llama 3.2 Vision. GLM-4.5-Air, Phi-4, Precog-24B, Codestral-22B and Qwen3-Coder-30B are listed without having been run: oklm info finds their headers loadable and their files did not fit on the authors' disk. Precog-24B, a story and role-play tune, is under general chat: the catalogue has no category of its own for it.

How a download is checked

  • Pinned. Each file is named by a URL at one revision of its repository (a commit, not a branch), with its size and SHA-256 compiled into oklm: what is downloaded tomorrow is what was listed.
  • Resumed. The bytes go to FILE.part; an interrupted pull asks for the rest with a Range request, and a connection cut in the middle is taken up again as long as bytes keep arriving. A server that answers the whole file instead starts the file again; a partial file larger than the file is dropped.
  • Verified. The file is renamed into place only when its size and its SHA-256 are the catalogue's; otherwise it is removed and the pull fails.
  • TLS. The certificate chain and the host name are checked against the system's authorities (on Windows, the system's root store). A build without OpenSSL cannot download from https and says so; the Windows build has OpenSSL once make windows-deps built it (build.md).
  • Redirects are followed (eight at most), to another host as well: Hugging Face answers a file with a redirect to its storage.

oklm_models_pull(), oklm_models_download() and the catalogue (oklm_catalog_get(), oklm_catalog_find()) are in include/oklm/client/models.h; oklm_client_get_data() in include/oklm/client/client.h is the streamed GET under them.

Maintaining the catalogue

tools/model_catalog.json is the list: a name, the categories, a title, one line of purpose, the licence, whether it is asked, whether the file was run, and each file's repository and path. A model whose file carries no chat template names its conversation format there ("chat_template": one of the names --chat-template takes, inference.md), read from the model's card and never guessed: vicuna for LLaVA 1.5 (its card: tuned from Vicuna, the prompt USER: ... ASSISTANT:), deepseek-coder for the two DeepSeek-Coder files (their tokenizer_config.json), mistral-v3 for Codestral 22B (Mistral's tokenizer_config.json; the quantizer's card prints another form) and foundation-sec for Foundation-Sec-1.1-8B-Instruct (its chat_template.jinja). The commands then chat with the model in that format without an option; StarCoder2 and Foundation-Sec-8B are completion models and name none. python3 tools/gen_model_catalog.py --local models pins each repository's current revision, reads the sizes and SHA-256 sums from the repository's listing, writes src/client/model_catalog.inc (committed) and compares every listed file it finds under models/ with its sum. python3 tools/docs/models_table.py --write then rewrites the table above. A model is marked as run only when the file the catalogue names was run: for the entries added on 2026-11-12 each was pulled with oklm models pull, run on the CPU and removed. Qwen3.8-27B is marked from the record of its run of 2026-11-13 (tests/model/test_infer_public.c against llama.cpp: the file of that SHA-256, removed since for lack of disk). Cyber-Ornith 1.5 9B and BugTraceAI-CORE-Ultra 27B were run the same day against llama.cpp and in oklm chat, then removed (compatibility.md, the qwen35 row, has the figures), and so were Muse Glimmer 30B and the three tunes of Qwen3.6-35B-A3B (the muse-glimmer and qwen35moe rows), one file on the disk at a time. Qwen2.5-VL 3B, 32B and 72B were fetched, run against llama.cpp and on both devices, and removed on 2026-11-14 (multimodal.md); their three entries were added to the generated table by hand in the generator's form (the revisions, sizes and sums the repositories listed that day, the sums those of the files run), so that the other entries' revisions did not move. The 72B's file is its IQ1_M, the one that was run: a disk with room takes a larger quantization of the same repository by its file name. MiMo-V2.6-Distill-Qwen-9B was fetched, run against llama.cpp and in oklm chat on the CPU, and removed on 2026-11-15 (compatibility.md, the qwen35 row); the generator ran whole that day and moved no other entry.