OKLMdocumentation v1.0.3
On this page

Speech synthesis

A text spoken into a sound: a language model writes audio codes from the text, a vocoder turns the codes into a waveform. This page is the audit of the published files OKLM runs (OuteTTS 0.2 and 0.3 with WavTokenizer's decoder, and Soprano with its own), what oklm speak and include/oklm/mm/speech.h do with them, the comparison with llama.cpp, and the limits. Speech recognition, the other way round, is in multimodal.md.

Status: implemented on the CPU: oklm speak, the library (oklm_speech_*, oklm_vocoder_*), the catalogue's speech category with outetts-0.2-500m, outetts-0.3-500m and soprano-1.1-80m, the server's route, the dashboard's voice and the chat window's (below); a speed, speaker files chosen by name on a server, FLAC files. English, one built-in voice. Soprano is the model that speaks faster than it is heard on a CPU (3 to 4 seconds of sound a second where OuteTTS makes 0.2, a first sound in under a second), with one fixed voice at 32 kHz; the sections before its own describe OuteTTS. Not there: a device path for the vocoder, other speech models, MP3 and Opus.

Using it

oklm models pull outetts-0.2-500m                       # 636 MiB, its licence asked (non-commercial)
oklm speak -m outetts-0.2-500m "Hello world, this is a test of speech synthesis."
oklm models pull outetts-0.3-500m                       # the same size; a licence that allows commercial use
oklm speak -m outetts-0.3-500m "Hello world, this is a test of speech synthesis."
oklm speak -m outetts-0.2-500m -o chapter.wav < chapter.txt
oklm speak -m OuteTTS-0.2-500M-Q8_0.gguf --vocoder WavTokenizer-Large-75-F16.gguf --voice speaker.json "Good morning."
oklm speak --vocoder WavTokenizer-Large-75-F16.gguf --codes codes.txt -o codes.wav
oklm speak -m outetts-0.2-500m --speed 1.25 --first-words 5 -o reply.flac "A first sound sooner, a little faster, in a FLAC file."
oklm models pull soprano-1.1-80m                        # 197 MiB, Apache-2.0: faster than real time on a CPU, one fixed voice
oklm speak -m soprano-1.1-80m "It costs $2.47 and arrives on Monday, Dr. Smith."

oklm speak writes a mono 16-bit WAV file at 24 kHz (32 kHz with Soprano, which takes the same options but --voice, --prompt-words and --no-guide; -o, speech.wav by default) from the text given, or read from standard input. -m takes the language model's GGUF file or the catalogue's name, whose vocoder is then found by itself; with a file, --vocoder names the vocoder. --voice FILE replaces the built-in voice by a speaker file of OuteTTS (below). --temp, --top-k, --repeat-penalty and --seed set how the codes are sampled, --no-guide leaves each word's first token to the model, --words N sets how many words one pass of the model speaks (24), --prompt-words N how many the model speaks from one prompt (64: the passes that hold in it share the prompt), --first-words N cuts the first pass shorter for an earlier first sound, --speed S (0.25 to 4) changes the pace and keeps the pitch (passes, prompts and the first sound, speed). A name ending in .flac given to -o writes a FLAC file, the same samples compressed without loss (FLAC). --codes FILE skips the language model and gives the vocoder audio codes read from a file (integers, 75 a second). --backend cpu|opencl|vulkan|auto says where the language model computes (cpu by default); the vocoder's network follows it there (on a device).

A server speaks for its clients with the same models: oklm serve --speak-model outetts-0.2-500m answers POST /v1/audio/speech (OpenAI's route) with a WAV file, or with raw samples sent pass by pass while they are made (server.md). The server's dashboard plays them: a Speak button on each reply of its chat, /speak, and a switch that reads every reply aloud, sentence by sentence while it is written (server.md); with a transcription model beside it, the dashboard holds a spoken conversation, the microphone listened to until a pause and the reply interrupted by speaking (server.md). A request names its voice among those the server offers (the one it was started with, the built-in one, the speaker files of its voice directories and of the model's folder: GET /v1/audio/voices), its speed, and wav, pcm or flac. The chat window (oklm-gui) has the same three, and plays the samples through the system's sound device: from its own server started with --speak-model, from a server at an address that has a voice, or with the library in the window itself, the model loaded at the first reading (gui.md). The window reads a reply while it is written, a sentence a call, and chooses among speaker files (~/.config/oklm/voices/*.json); with its Talk box it holds a spoken conversation, the microphone listened to until a pause and the reply interrupted by speaking (gui.md).

While it runs the command says how much is spoken, then how fast it was:

$ oklm speak -m outetts-0.2-500m -o cat.wav "The catalogue name finds both files."
[2.6 s of speech after 6.0 s]
first sound after 6.00 s; 1 prompt of 901 tokens (0 kept in the cache) in 1.21 s, 262 tokens written in 4.61 s, the vocoder 0.18 s
cat.wav: 2.60 s of speech in 6.00 s (0.43 s of sound per second)
$ oklm transcribe -m whisper-base cat.wav
The catalog name finds both files.

The second line is oklm_speech_last_stats(): when the first pass was handed over, the prompts evaluated and their tokens, those the context's cache already held, the tokens the model wrote, and the time of each part.

The two files

Those llama.cpp's tts example names (--tts-oute-default):

Language model Vocoder
Repository OuteAI/OuteTTS-0.2-500M-GGUF ggml-org/WavTokenizer (converted from novateur/WavTokenizer-large-speech-75token)
File OuteTTS-0.2-500M-Q8_0.gguf WavTokenizer-Large-75-F16.gguf
Size 536,666,752 bytes 130,186,688 bytes
SHA-256 2b23a91e20202f1e4a5904078c69a01bc8a404b8b0f8477c0f7a0c61eb6ac23a 2356baa8631cc2995ea3465196a017a2733600d849a91180c0f97fa7fb375bbe
general.architecture qwen2 (Qwen2.5-0.5B retrained) wavtokenizer-dec
Shape 24 blocks, width 896, 14 heads over 2 key/value heads, FFN 4864, RoPE base 1e6, context 32768 code width 512, stacks of width 768, FFN 2304, output rows of 1282, context 8192
Tensors 290 (499 M parameters; Q8_0, the norms and biases F32) 161 (matrices and convolutions F16, norms and biases F32)
Vocabulary 157,696 tokens, byte-level BPE with the qwen2 pre-tokenizer, no BOS added, EOS <|im_end|> (151645) 4096 codes (tokenizer.ggml.model = none)
Licence CC-BY-NC-4.0 (the model card; the catalogue asks for agreement) MIT (general.license, and the source repository's card)

Other quantizations of the language model are published in the same repository, from Q2_K (344 MB) to F16 (1.0 GB); Q4_K_M is 403 MB. The vocoder also exists as Q5_1 (73 MB), which OKLM's vocoder does not read (F32 and F16 only). The language model needs nothing new: OKLM's qwen2 runtime loads it (oklm info reads it, oklm run generates from it), and the audio tokens are ordinary vocabulary entries.

OuteTTS 0.3 is the catalogue's outetts-0.3-500m, with the same vocoder file:

Language model
Repository, file OuteAI/OuteTTS-0.3-500M-GGUF, OuteTTS-0.3-500M-Q8_0.gguf
Size, SHA-256 536,666,880 bytes, e6cb54453b2de1eaee85bd46b67de9c8cdd3c2dfe117cc3e3da794ddc64dad85
Shape the same qwen2 model: 24 blocks, width 896, 290 tensors, 499 M parameters, Q8_0
Vocabulary 157,696 tokens: 0.2's (<|text_sep|>, <|code_start|>, <|code_end|> and the codes keep their ids), with <|space|> (156777) and punctuation marks (<|period|> 156779, <|comma|> 156782, ...) among the entries 0.2 left unused
tokenizer.chat_template outetts-0.3 (what llama.cpp's example reads the version from)
Licence CC-BY-SA-4.0 (the model card): commercial use allowed, changes shared alike

The text and the prompt

The text is normalized as OuteTTS's prompt processor does, with numbers spelled as llama.cpp's tts example spells them (OuteTTS itself uses the inflect package; romanization of Chinese, Japanese and Korean with uroman and MeCab is in neither llama.cpp nor OKLM):

  1. each run of digits, with its decimals when a dot and digits follow, is spelled in English: 42 is forty two, 3.14 is three point one four, 1234567 is one million two hundred thirty four thousand five hundred sixty seven; above 2,147,483,647 the digits are read one by one (llama.cpp drops such a number). Nothing separates the spelled number from letters around it: 2nd is twond, and 1,000 is one zero, in both;
  2. upper case becomes lower case (ASCII only);
  3. -, _, /, ,, . and \ become spaces;
  4. every character that is not a letter a to z or white space is dropped, accented letters included (café is caf);
  5. the words are separated by <|text_sep|>.

The prompt of one pass, the voice being a recording's words with the duration and the audio codes of each:

<|im_start|>
<|text_start|>the<|text_sep|>overall<|text_sep|>...<|text_sep|>lovely<|text_sep|>hello<|text_sep|>world<|text_end|>
<|audio_start|>
the<|t_0.08|><|code_start|><|257|><|740|><|636|><|913|><|788|><|1703|><|code_end|>
overall<|t_0.36|><|code_start|><|127|><|201|>...<|code_end|>
...
lovely<|t_0.56|><|code_start|><|634|>...<|1677|><|code_end|>

The voice's words come first in the text, then the words to speak; the audio part holds the voice's words only, and the model continues it: a line feed, then each word of the text as its letters, a duration token <|t_S.SS|>, <|code_start|>, its codes, <|code_end|>, a line feed between words, and <|audio_end|>, a line feed and <|im_end|> after the last. The model was trained on exactly this layout (get_training_prompt() of OuteTTS's prompt_processor.py).

OuteTTS 0.3's layout separates the words by <|space|>, which also ends a word's codes, and has no <|code_start|>:

<|im_start|>
<|text_start|>the<|space|>overall<|space|>...<|space|>lovely<|space|>hello<|space|>world<|text_end|>
<|audio_start|>
the<|t_0.08|><|257|><|740|><|636|><|913|><|788|><|1703|><|space|>
overall<|t_0.36|><|127|><|201|>...<|space|>

OKLM reads the layout from the vocabulary (a model that has the token <|space|> is 0.3; oklm_speech_format()), builds the prompt the way llama.cpp's example derives it from 0.2's (<|text_sep|> becomes <|space|>, <|code_start|> goes, <|code_end|> becomes <|space|>), and counts a word written at <|space|>. The same voice, the same normalization, the same sampling and guide. OuteTTS's own prompt processor for 0.3 keeps the punctuation as tokens (<|period|>, ...); llama.cpp's example does not, and neither does OKLM: no punctuation reaches the model. The prompt is 30 tokens shorter (858 before the first word of the text: no <|code_start|> for the voice's 30 words).

The voice. The built-in one is en_male_1, a default speaker of OuteTTS, as llama.cpp's example embeds it: 30 words ("the overall package from just two people is pretty remarkable ..."), 658 codes, 8.8 seconds. It is compiled in (src/mm/speech_voice.inc). --voice (and oklm_speech_params.voice) reads a speaker file of OuteTTS instead: a JSON object with words, an array of {"word": "...", "duration": 0.24, "codes": [..]}, as interface.save_speaker() of the outetts package writes it. Making such a file from a recording needs WavTokenizer's encoder and a word aligner, which are not here: the outetts package does it. A voice costs its codes in every prompt: the built-in one makes the prompt 888 tokens long before the first word of the text.

The tokens. Everything between <| and |> above is one token:

Tokens Ids
<|im_start|>, <|im_end|> 151644, 151645
<|text_start|>, <|text_end|>, <|audio_start|>, <|audio_end|> 151665 to 151668
<|code_start|>, <|code_end|>, <|text_sep|> 151669, 151670, 151671
the audio codes <|0|> to <|4099|> 151672 to 155771
the durations <|t_0.00|>, <|t_0.01|>, ... from 155772

An audio code is told from the rest by its id: token 151672 + c is code c. The vocabulary has 4100 of them and the vocoder's codebook 4096: OKLM finds the first by its name (<|0|>), checks that the codes are consecutive, and keeps the codes the vocoder knows. (llama.cpp's example keeps ids 151672 to 155772, one duration token included, and would give the vocoder a code above 4095 if the model wrote one.)

Sampling and the end. llama.cpp's example samples among the 4 most likely tokens at temperature 1 (top_k = 4, no other sampler), up to 4096 tokens, and stops at an end-of-generation token. The model card advises temperature 0.1 with a repetition penalty of 1.1. OKLM's default is llama.cpp's (see what is heard for why), with the guide tokens llama.cpp has as an option (--tts-use-guide-tokens) turned on: the first token of each word is set to the text's own, which keeps the model on the text. A pass ends at <|audio_end|> or the end token, when as many words are written as the text has, or after 160 tokens per word.

Passes. A text is cut into passes of at most 24 words (--words): a sentence each, and a sentence that is too long alone after a comma, a semicolon or a colon, then anywhere. Sentences are not joined in a pass, however short: a sentence's end is then always a pass's end, where the pause between two sentences is made (below). Each pass's sound is handed over as soon as it is made (the callback of oklm_speech_synthesize_chunks(), a chunk of the server's stream). How the passes share prompts is the next section.

Passes, prompts and the first sound

What a prompt costs, and what can be kept of it. The prompt of OuteTTS is the header, the voice's words, the text's words, then the voice's codes: with the built-in voice 62 tokens before the text's first word and 826 after its last. A language model's cache holds what comes before a change: from one text to another, the 62 tokens of the header and the voice's words stay valid; the voice's 658 codes, which are most of the prompt, come after the text and have to be evaluated again with every new text. Keeping "the voice" in the cache is therefore not possible with this layout, which is the one the model was trained on; what is possible is to evaluate fewer prompts.

One prompt for several passes. The passes of a text are written one after the other in the same context while they hold 64 words together (--prompt-words, oklm_speech_options.prompt_words; fewer when the context is smaller or the voice longer: 48 tokens are counted per word): their words are all in the prompt's text, and the model goes on from one pass's last word to the next pass's first as it does inside a pass. The prompt is evaluated once per group instead of once per pass, and the codes are the ones a single pass of all those words would give. Each pass is given to the vocoder when its last word is written:

  • the vocoder reads the pass with the 75 codes before it (a second) as context, whose samples are dropped;
  • the last 8 codes of a pass that is not the group's last wait for the next pass, which gives the vocoder what follows them;
  • a pass begins by a cross-fade of 10 ms from what the vocoder wrote there with the pass before;
  • a pass that ends a sentence (., !, ? or a line feed) is given whole, nothing waiting, with 0.3 second of silence after it: the model is given no punctuation and pauses little between two sentences by itself, where a new prompt begins with a pause.

The samples are not those of one call of the vocoder on all the codes: its group normalizations and its attention read every frame they are given, so a pass decoded with a second of context differs from the same codes decoded with the whole text, as two passes of their own always did (57 words in 11 passes against one call: 14.4 dB of signal to difference; in 4 passes 17.2 dB). What the joins are held to is continuity: the largest step between two samples is 0.265 in the 11 passes and 0.258 in the one call, and Whisper base hears the same words in both. The quarter of a second silenced at the start is silenced once per prompt. When the model ends a group early (its end token before the last word, or a context that is full), the pass it was in is given as it is and the passes after it get a prompt of their own. With --prompt-words equal to --words every pass has its own prompt, as before.

From one prompt to the next, and from one call to the next (a server's requests), the tokens the cache already holds at the start of the new prompt are kept: 62 with the built-in voice, 70 when a text begins with the same word, the whole prompt but its last token when the same text is spoken again (another seed, another speed).

The first sound. It comes after the prompt's evaluation and the codes of the first pass. --first-words N (first_words) cuts the first pass at N words, inside a sentence when it must: since the passes of a group share a prompt and are joined as above, a short first pass costs no prompt of its own and no audible seam. The chat window used to ask for a reply's first five words as a synthesis of their own, at the price of a second prompt and of a break in the voice between the two.

The short first pass is a piece of the first sentence, or that sentence whole when it has N words or fewer: it moves no sentence's end, and the 0.3 second of silence comes after every sentence that another follows under the same prompt, with the option and without it. A text is therefore as long either way. (Until 2026-11-15 sentences that held together in 24 words were one pass, with no silence between them, and a text of short sentences was 0.3 second longer with --first-words than without.) The tests' tiny model (two codes a word, four samples a code) shows it in bytes: "Hello there, world. One two three." is a WAV file of 14 540 bytes, 48 samples of words and 7200 of silence after the third word, whether --first-words is absent, 2 (a first pass that ends at the comma: three passes) or 5 (two passes) (test_speech_route). The published OuteTTS 0.2 on the 64-word text of the table below, its language model on OpenCL (RX 7900 XT): 1 016 684 bytes, 21.18 s of speech, 2 prompts of 1913 tokens both ways; five passes and a first sound after 0.87 s without the option, six passes and a first sound after 0.58 s with --first-words 5; the whole text after 5.8 s (the table below has the CPU and Vulkan).

A model that ends the sound at a sentence's end leaves the sentences after it to a prompt of their own, as before; they used to be lost when they were in the same pass. OuteTTS 0.3 does so on "Hello. Yes.": with the default sampling the commit before this cut (built and run again on 2026-11-15) gave 1.71 s of sound under one prompt, in which Whisper large-v3 hears "Hello.", and the code since gives two prompts and 2.77 s, heard "Hello. Yes.". At temperature 0 the same text shows the second prompt and nothing more: 0.3 then writes codes that are close to silence, before the cut (one prompt, 107 tokens, 1.32 s) as after it (two prompts of 1720 tokens, 156 tokens written, 2.22 s), and Whisper base, large-v3 and large-v3-turbo hear no word in either (test_published_03 checks the prompts and the pause, not what is heard).

Measured on 2026-11-15, with every sentence a pass of its own: a text of five sentences and 64 words, 20.8 to 22.1 seconds of speech, OuteTTS 0.2 Q8_0. On the CPU, OKLM_THREADS=4 on the Core i7-11700K while another job kept the load average at 3.1 to 5.1; on the RX 7900 XT the language model and the vocoder's network on the device (load 1.1 to 2.9). Two or three runs of each, one after the other: the token counts are the same from run to run, the times within the ranges given.

Backend Prompts Their tokens (kept in the cache) Time in the prompts Tokens written Sound First sound after The whole text after
--prompt-words 24: a prompt per pass, its start kept CPU 4 3687 (210) 4.1 s 1913 21.39 s 6.5 to 6.6 s 43.5 to 43.7 s
OpenCL 4 3687 (210) 0.19 s 1923 21.53 s 0.89 to 0.92 s 5.9 to 6.0 s
Vulkan 4 3687 (210) 0.16 to 0.17 s 1967 22.11 s 0.69 to 0.70 s 4.65 to 4.67 s
the default: the passes that hold in 64 words under one prompt CPU 2 1913 (70) 2.2 s 1828 20.86 s 6.6 s 40.9 to 41.0 s
OpenCL 2 1913 (70) 0.10 s 1852 21.18 s 0.86 to 0.87 s 5.76 to 5.80 s
Vulkan 2 1913 (70) 0.09 s 1820 20.75 s 0.65 to 0.66 s 4.31 to 4.34 s
the same with --first-words 5 CPU 2 1913 (70) 2.2 s 1828 20.86 s 4.6 s 40.8 to 40.9 s
OpenCL 2 1913 (70) 0.10 s 1852 21.18 s 0.57 to 0.58 s 5.75 to 5.78 s
Vulkan 2 1913 (70) 0.09 s 1820 20.75 s 0.44 s 4.34 to 4.35 s
the same with --first-words 8 --words 12 (eight passes) CPU 2 1913 (70) 2.2 s 1828 20.86 s 6.6 to 6.8 s 41.1 to 41.2 s
OpenCL 2 1913 (70) 0.10 s 1852 21.18 s 0.86 to 0.88 s 5.79 to 5.84 s
Vulkan 2 1913 (70) 0.09 s 1820 20.75 s 0.65 s 4.35 s

(The text is 64 words once its number is spelled, two more than the 62 a context of 4096 tokens is counted to hold with this voice: two prompts. Its first sentence has eight words: the first sound without --first-words comes after that sentence, and --first-words 8 changes nothing to it. The sound is the same with and without --first-words on a backend, to the byte: 1 001 324 bytes on the CPU, 1 016 684 on OpenCL, 996 204 on Vulkan; it differs from one backend to another as on a device says, and with the prompts, which are other contexts. The series this table replaces was made before the sentences were passes of their own, under a load three times heavier: its first pass was the first 24 words, not the first sentence.)

The prompts are a small part of the time: on the CPU the 1913 tokens of the two prompts are evaluated in 2.2 seconds, where the model writes 49 tokens a second one at a time, and a second of speech is about 88 tokens. One prompt for the group saves what the other prompts cost (1.9 seconds of 43 on this text); the first sound comes earlier with a short first pass, by the tokens of the first sentence's other words (6.6 s to 4.6 s); and neither makes the synthesis faster than it is heard: on this CPU the language model writes a second of speech in two, which only a device (3.7 seconds of sound a second on OpenCL, 4.8 on Vulkan) or a smaller model changes: Soprano.

Heard. Six sentences of 57 words as one text, Whisper base listening (the table of listening by proxy spoke them one by one):

Seed A prompt per pass (three prompts) One prompt, a first pass of 6 words
1234 56 of 57 ("vocoder" heard "bar2"), 17.13 s 55 of 57 ("on" heard "a", "vocoder"), 14.93 s
7 56 of 57 ("vocoder"), 16.81 s 52 of 57 ("quick" heard "quit", "and arrives on" heard "in a rinds and", "vocoder"), 14.89 s
99 53 of 57 ("and arrives on" heard "in a rites and", "vocoder"), 16.29 s 55 of 57 ("vocoder turns" heard "vacay returns"), 15.37 s
Words right 165 of 171 (96.5%) 162 of 171 (94.7%)

("sea shells" heard "seashells" counted right, "$42" as "forty two dollars".) The same slip ("and arrives on") is heard under both, with different seeds: three words between the two columns are within what a seed changes. Under one prompt the model pauses less between sentences (it is given no punctuation, and nothing but a line feed separates two words), so the same text was two seconds shorter in these runs, made before the 0.3 second of silence after a sentence was added: that silence gives the pause back between passes, and changes no code.

The vocoder

WavTokenizer's decoder is a Vocos network with an attention stack in front; llama.cpp's converter names its tensors as below. The activations are [channels, frames], one frame per code.

  1. Codes to features: row c of token_embd.weight (4096 x 512).
  2. Input convolution: conv1d (kernel 7, 512 to 768 channels, half padding).
  3. "posnet", six blocks, their kind told by the tensors they have:
    • blocks 0, 1, 3, 4, ResNet: group normalization (norm1: 32 groups over the channels of a group and every frame, epsilon 1e-6), swish (x * sigmoid(x)), conv1 (kernel 3, 768 to 768), norm2, swish, conv2, and the input added;
    • block 2, attention: attn_norm (group normalization), three 1x1 convolutions attn_q, attn_k, attn_v, one head over every frame in both directions with the scores divided by sqrt(768), attn_output (1x1), and the input added;
    • block 5: attn_norm alone, a group normalization.
  4. token_embd_norm: a layer normalization over the channels (epsilon 1e-6).
  5. ConvNeXt, twelve blocks (convnext.N): a depthwise convolution dw (kernel 7), a layer normalization norm, a linear layer pw1 (768 to 2304), GELU, pw2 (2304 to 768), a per-channel scale gamma, and the input added.
  6. output_norm (layer normalization) and output, a linear layer to 1282 values per frame: the log-magnitudes of 641 frequency bins, then their phases.
  7. The inverse short-time Fourier transform (oklm_istft()): the magnitude is exp() of the first half, at most 100; the complex spectrum goes through an inverse real transform of 1280 points; each frame is multiplied by a periodic Hann window of 1280 samples; the frames are added 320 samples apart and the sum divided by the sum of the squared windows; 480 samples are dropped at each end ("same" padding). A code is 320 samples: 75 codes a second at 24 kHz (the rate is not in the file: it is the model's, and llama.cpp's example writes it as a constant too).

src/mm/vocoder.c binds the tensors where the file is mapped (the matrices in F16, multiplied as they are; biases, norms, the code table and the depthwise kernels as F32), and runs the convolutions as products over unfolded frames with oklm_matmul(), the attention with oklm_attention(), the layer normalizations, swish and GELU with the core ops, as the Whisper encoder does; the group normalization, the depthwise convolution and the inverse transform (a mixed-radix transform: 1280 = 2^8 x 5) are plain C. The transform's frames are computed on the worker team, 64 at a time, and added to the sound in their order (the samples of one thread, to the bit: 52 ms became 9 ms for 6.4 seconds of sound). Two choices differ from llama.cpp, both towards the model's definition in PyTorch:

  • GELU is the exact one (erf), as torch.nn.GELU(); ggml computes the tanh approximation through a half-precision table;
  • the inverse real transform is the true one. llama.cpp's example sums the one-sided spectrum and divides by 641, which scales the signal by 1280/1282 and adds the constant and the highest bin a second time.

Comparison with llama.cpp

The reference. The pinned llama.cpp of build/llama.cpp (8642541b) still has the wavtokenizer-dec architecture, but its llama-tts was rewritten for other models in August 2026 and no longer runs OuteTTS. The reference is the last commit before that, built for the CPU in a scratch directory: 1c3c9674de4d455f1e571bed808252af54932767 (2026-08-04), cmake -DGGML_NATIVE=ON -DLLAMA_CURL=OFF, target llama-tts, with a few lines added to tools/tts/tts.cpp that write the prompt's ids, the generated ids, the vocoder's output rows and the samples to files. It ran with --top-k 1 (the most likely token each time) on three sentences: "Hello world, this is a test of speech synthesis.", "The quick brown fox jumps over the lazy dog." and "It costs 42 dollars and arrives on Monday morning."

The prompt's token ids: identical for the three sentences (906, 906 and 913 ids).

The audio codes. Chosen greedily by each program, they are the same for the first 75, 104 and 29 codes, then differ: the codes are close calls by nature (the model is meant to be sampled among four), and the two programs do not round alike. Fed llama.cpp's own tokens, OKLM's most likely token is llama.cpp's at 258 of 262 steps, 252 of 254 and 289 of 300; where it is not, OKLM's logits of the two tokens differ by 0.033 at most (0.007 and 0.019 for the other two sentences).

The vocoder, on llama.cpp's codes (212, 204 and 240 of them). The third implementation is tools/gen_speech_fixtures.py --run: numpy in double precision, written from the model's definition.

Compared, sentence 1 Largest difference Mean difference Signal to difference
Rows (log-magnitude, phase): OKLM against numpy 1.1e-4 6.3e-7 131.2 dB
Rows: llama.cpp against numpy with ggml's GELU 6.5e-2 8.4e-4 71.1 dB
Rows: OKLM against llama.cpp 4.9e-1 3.3e-3 57.4 dB
Rows: OKLM built with ggml's GELU, against llama.cpp 6.5e-2 8.4e-4 71.1 dB
Samples: OKLM against numpy (irfft) 1.1e-6 2.5e-8 124.7 dB
Samples: OKLM against llama.cpp 4.4e-3 5.6e-4 40.9 dB
Samples: llama.cpp's transform applied to OKLM's rows, against llama.cpp 3.6e-3 1.4e-4 50.8 dB
Samples: the true transform against llama.cpp's, both on llama.cpp's rows 3.6e-3 5.3e-4 41.5 dB

The samples are in [-1, 1] with a root mean square of 0.09. Sentences 2 and 3 give 51.7 and 49.4 dB on the rows against llama.cpp (70.5 and 65.9 dB with ggml's GELU) and 39.4 and 39.6 dB on the samples. Read together: OKLM's network is the definition's to 131 dB; llama.cpp's is the same network to 71 dB, the rest being its half-precision GELU table; the 41 dB between the two programs' samples is the difference between the two inverse transforms, which the last line measures alone.

OuteTTS 0.3 was not run against llama.cpp: the build of commit 1c3c9674 was a scratch build, and the pinned llama.cpp no longer speaks OuteTTS. What stands in for it: the source of that commit's tools/tts/tts.cpp (the version read from the chat template, the separator in process_text() and in the guide, the three replacements applied to the built-in voice, audio_text_from_speaker() and audio_data_from_speaker() for a speaker file), which the test follows (0.3's prompt is 0.2's after those replacements, for the built-in voice and a speaker file); the prompt of the first sentence is 876 tokens, the 906 that are llama.cpp's for 0.2 less the 30 <|code_start|>; the vocoder is the same file; and what is heard, below.

Listening by proxy

Six sentences spoken by oklm speak, transcribed by oklm transcribe with Whisper, and the words compared (lower case, numbers spelled; "$42" read as "forty two dollars"). The default settings, seed 1234:

Sentence Heard by Whisper base Heard by Whisper small
Hello world, this is a test of speech synthesis. the same, 9 of 9 the same, 9 of 9
The quick brown fox jumps over the lazy dog. "... over the lazy dobs.", 8 of 9 the same, 9 of 9
It costs 42 dollars and arrives on Monday morning. "It costs $42 and arrives on Monday morning.", 10 of 10 the same words, 10 of 10
Please open the window before you leave the room. the same, 9 of 9 "PZAP in the window before you leave the room.", 7 of 9
The model writes audio codes and the vocoder turns them into sound. "... and the VARTA turns them into sound.", 11 of 12 "... and the Vita turns them into sound.", 11 of 12
She sells sea shells by the sea shore. "She sells seashells by the sea shore.", 6 of 8 "She sells seashells by the seashore.", 4 of 8
Words right 53 of 57 (93.0%); the first three sentences 27 of 28 50 of 57 (87.7%); the first three 28 of 28

"seashells" and "seashore" are the same sounds written as one word: counted right, the figures are 55 and 54 of 57. What is left is one ending ("dog"), the start of one sentence for one of the two listeners, and "vocoder", a word neither the speech model nor Whisper knows.

The same six sentences with other settings, heard by Whisper base:

Settings Words right
temperature 1, top-k 4 (llama.cpp's; the default), with or without the guide 53 of 57 (93.0%)
temperature 0.4, top-k 40, repeat penalty 1.1, guide 46 of 57 (80.7%)
the most likely token, no guide 45 of 57 (78.9%)
temperature 0.1, repeat penalty 1.1 (the model card's), no guide 42 of 57 (73.7%)
the default with seed 7, heard by Whisper small 49 of 57 (86.0%)
the same without the guide, heard by Whisper small 42 of 57 (73.7%)

At low temperatures the model slurs words ("and arrives on" heard as "in a rise on") and, without the guide, sometimes begins by repeating the voice's last words ("and it looks lovely. Please open the window before you"): the repeat penalty and a low temperature both work against a model whose codes repeat by nature. With seed 1234 the guide changed nothing; with seed 7 the run without it began one sentence with the voice's last words again, which is what the guide is for. "sea shells" heard as "seashells" counts as two words wrong in every row (four with "seashore" for Whisper small).

OuteTTS 0.3, the six sentences as one text under one prompt (seed 1234, a first pass of 6 words), heard by Whisper base: "Hello world, this is a test of speech synthesis. The quick brown fox jumps over the lazy dog. It costs $42 and a rise on Monday morning. Please open the window before you leave the room. The model writes audio codes and a vaker that turns them into sound. She sells seashells by the seashore.": 54 of 57 ("arrives", "the", "vocoder"), where 0.2 gave 55 with the same settings (above); with seed 7, 52 of 57 ("quid" for "quick", "door" for "dog", "Peace out in" for "Please open", "vocoder"), as 0.2 with that seed. 0.3 speaks slower: 19.5 s and 19.2 s for 0.2's 14.9 s, 1658 and 1634 tokens for 1417. In one run of the 64-word text with a first pass of 5 words, its first word was not heard ("Synthesis turns a written text into sound ..."): 0.3 can begin to speak inside the quarter of a second silenced after a prompt, which is llama.cpp's for both versions and was kept.

A longer text, five sentences of 64 words, took four passes (7.7, 4.3, 5.1 and 3.7 seconds of speech; 20.8 seconds in all) and came back from Whisper small as "Speech synthesis turns a written text in the sound, the language model reads the words and writes audio codes 75 for each second. The Vata then turns the codes into a waveform at 24 kHz. A long text is cut into passes of a few sentences and each pass is spoken on its own. This paragraph has five sentences and takes several passes.": "into sound" and "vocoder" misheard, the rest right.

Limits

  • English only. The model card lists Chinese, Japanese and Korean as experimental; they need a romanization OKLM does not have, and the normalization drops every character outside a to z. OuteTTS 0.3's card adds French and German. Tried once with what is here (the built-in English voice, texts without accents, Whisper small told the language): of "Guten Morgen, das ist ein Test der Sprachsynthese. Das Wetter ist heute sehr gut, und wir gehen in die Stadt." the first 13 words of 19 are heard and the end is not ("um die Gaming gestartet"); of a French sentence of 17 words, 5. Another language takes a voice recorded in it, the accented letters kept as OuteTTS's own processor keeps them, and numbers spelled in it: not done.
  • One built-in voice (a man's, en_male_1). Other voices are speaker files made elsewhere; OKLM cannot make one from a recording: cloning a voice takes WavTokenizer's encoder (OKLM has its decoder only) to turn the recording into codes, and an aligner to cut them word by word. The chat window lists the speaker files it finds and speaks with the one chosen (gui.md). A server offers the voice it was started with (--speak-voice), the built-in one and the speaker files of its voice directories (--speak-voices) and of the model's folder: a request names one (server.md), and oklm_speech_set_voice() changes the voice of an open synthesizer.
  • No pronunciation control: no punctuation reaches the model, so no pause is asked for inside a pass; abbreviations, symbols and digits stuck to letters are read as their letters allow.
  • Length. A text of any length is spoken, pass after pass. One pass takes the prompt (888 tokens with the built-in voice, two or so per word of the text) and what the model writes (about 75 tokens per second of speech and four per word) in a context of 4096 tokens: about 40 seconds; --words takes up to 64. The default of 24 words keeps a pass near 10 seconds, about the length of the voice's own recording. The vocoder takes 8192 codes a call (109 seconds).
  • Not streamed inside a pass. oklm_speech_synthesize_chunks() gives each pass's samples when the pass is done; the first sound comes after the prompt and the first pass's codes. A client that wants it early asks for a short first pass (first_words), which costs no prompt (above). The vocoder's attention reads every frame it is given, so the end of a pass is written with the few codes after it, or with none when it is the last.
  • The voice's codes are evaluated with every prompt: they follow the text in the prompt, so the cache keeps the 62 tokens before the text and no more. A shorter voice makes a shorter prompt.
  • Slower than real time with OuteTTS (Soprano is not) on this machine's CPU (Core i7-11700K, 8 cores). The best run spoke 2.6 seconds in 6.0 seconds, 0.43 second of sound per second; the runs of the tables above, made on 2026-11-15 with four threads while another job kept the load at 3 to 5, gave 0.49 to 0.51 (the series before them, while other builds held every core, 0.12 to 0.38). Nearly all of it is the language model: the prompt's 890 tokens once per group of passes (about a second in those runs), then about 90 tokens per second of speech, written at 49 a second; the vocoder and the inverse transform took 1.1 second for 20.9 seconds of sound. llama.cpp's example, run in turn with OKLM under that load, took 28 to 74 seconds for the sentence OKLM spoke in 19 to 25: the same order, the load deciding. Peak memory was 1.65 GiB.
  • The vocoder's inverse transform on the CPU. --backend opencl|vulkan puts the language model on a device, where nearly all the time was, and the vocoder's network with it (on a device).
  • Licence: OuteTTS 0.2 is CC-BY-NC-4.0, not for commercial use; OuteTTS 0.3 is CC-BY-SA-4.0, which allows it.
  • A sampled run is repeatable with its seed on one machine, not across machines: near-equal logits can round differently.

Speed

--speed S, oklm_speech_options.speed and the route's speed (0.25 to 4; 1 is the model's own pace) change how long the sound lasts and keep its pitch. OuteTTS has no control of its pace (the durations in its prompt are the voice's, and the model writes those of the text itself), so the change is made on the samples, pass by pass, by oklm_speech_stretch(): waveform similarity overlap-add. Frames of 40 ms are taken from the input every 20 ms times the speed, each moved by at most 10 ms to the place where it is most like what would have followed the frame before (the largest normalized correlation), and added 20 ms apart under a Hann window. The first frame is the input's own start, so two passes still join.

A 220 Hz tone comes out at 220 Hz (216 to 224 counted on its zero crossings) at speeds 0.5, 0.75, 1.5, 2 and 4, at 0.85 to 1.15 of its level and of the length asked to the sample (the test). The 64-word text above at 1.5 is 14.12 s of sound for 21.18 s, at 0.75 28.24 s (OpenCL, 2026-11-15, the 0.3 s after each sentence stretched with it); Whisper base heard the same words in the three before those pauses were added. Above 2 speech is hard to follow: the frames skip more than they hold.

FLAC

src/mm/flac.c (include/oklm/mm/flac.h) writes FLAC files of mono 16-bit sound without a library: the stream marker, a STREAMINFO block, frames of 4096 samples (the last one shorter), each one subframe: a constant, the samples as they are, or one of the format's five fixed predictors (orders 0 to 4, the one whose residual is smallest) with its residual in Rice codes, cut into the 1 to 256 partitions that take the fewest bits, each with its own parameter; the two checksums of every frame. No linear prediction (LPC) and no MD5 of the samples (the header's field is zero, which the format reads as "not computed": flac -t says so and checks the rest).

  • oklm speak -o NAME.flac, the route's "response_format": "flac" (whole, its length in the header, or streamed frame by frame with "stream": true, the length left unknown), oklm_flac_encode() and oklm_flac_writer in the library.
  • Checked with the reference tools: flac -t accepts the files, flac -d gives back the WAV file oklm speak writes byte for byte, and ffmpeg decodes the same samples. 5.3 s of the vocoder's sound: 151,235 bytes for the 256,044 of the WAV file (59%), where flac -8, with linear prediction, makes 142,653 (56%).
  • oklm_flac_decode() reads what the encoder writes (one channel, 16 bits, those subframes), for the tests and for reading a file back; it is not a general FLAC reader (Allegro is, for oklm transcribe).

MP3, Opus and AAC are not written, and the route refuses them (400, saying so). They are lossy codecs whose encoders are whole programs: MP3 needs a polyphase filter bank, an MDCT, a psychoacoustic model and its bit allocation loops and Huffman tables; Opus is two codecs (SILK and CELT) and a range coder, some tens of thousands of lines in its reference implementation. Neither fits a contained, tested module of this size, and OKLM links no codec library to write sound. FLAC is the compressed form: about half the size of the WAV, exact, read by browsers (<audio>, decodeAudioData()), ffmpeg and every player.

Soprano

Soprano 1.1 (ekwek/Soprano-1.1-80M, Apache-2.0) is the model that speaks faster than it is heard on a CPU. It has no voice in its prompt, so none of the cost measured above, and it writes 16 tokens for a second of speech where OuteTTS writes 90. English, one fixed voice, 32 kHz.

oklm models pull soprano-1.1-80m                        # 197 MiB, Apache-2.0
oklm speak -m soprano-1.1-80m "Hello world, this is a test of speech synthesis."
oklm speak -m soprano-1.1-80m --first-words 5 -o reply.flac < reply.txt
oklm speak -m Soprano-1.1-80M-soprano-tok.F16.gguf --vocoder codec-F32.gguf "Dr. Smith paid $2.47 at 8:05."
oklm serve -m qwen3-8b --speak-model soprano-1.1-80m    # POST /v1/audio/speech at 32 kHz
$ oklm speak -m soprano-1.1-80m -o cat.wav "The catalogue name finds both files."
[2.3 s of speech after 0.3 s]
first sound after 0.27 s; 1 prompt of 29 tokens (0 kept in the cache) in 0.01 s, 37 tokens written in 0.16 s, the vocoder 0.10 s
cat.wav: 2.30 s of speech in 0.27 s (8.42 s of sound per second)
$ oklm transcribe -m whisper-base cat.wav
The catalog name finds both files.

(The best run seen, four threads; the same sentence took OuteTTS 0.2 6.00 s at its best, above.)

oklm speak, the library and the route are those of OuteTTS; what a caller sees change is the sample rate (oklm_speech_rate() is 32000), the voice (none to choose: --voice, oklm_speech_params.voice and oklm_speech_set_voice() with a file are refused, saying "Soprano speaks with one fixed voice: a speaker file does not apply") and the sampling's defaults (below).

Its two files

Language model Decoder
Original ekwek/Soprano-1.1-80M (revision 27b5a5f5): model.safetensors, 159,449,112 bytes, bfloat16 the same repository's decoder.pth, 121,448,447 bytes, SHA-256 11245487e4588b0f4128d6b4c82035212bad42ffb26f36e059e9c7f861c1f8c4: a PyTorch checkpoint
What the catalogue installs BricksDisplay/Soprano-1.1-80M-GGUF (revision d4f2e374), Soprano-1.1-80M-soprano-tok.Q8_0.gguf the same repository's codec-F32.gguf
Size, SHA-256 84,921,376 bytes, b16d98166e83b3e5f4740b25e5b14f09346dffd4ffab1660ac787768aaf414a5 121,423,904 bytes, 6004d7a8cbd04a6ae14172a6b78dfac2114e4a75a5dacb7ed3a454ea5ec329a1
general.architecture qwen3 soprano (the layout of codec.cpp's converter)
Shape 17 blocks, width 512, 4 heads over 1 key/value head of 128, FFN 2304, RoPE base 10000, context 1024 hidden states of width 512 in, an input layer to 768 channels, 8 ConvNeXt blocks (depthwise kernel 3, FFN 2304), a head of 2050 values per frame; n_fft 2048, hop 512, 4 frames per state
Tensors 190 (79.7 M parameters; Q8_0, the norms F32) 81 (30.4 M values, F32)
Vocabulary 8192 tokens: [UNK], [TEXT], [START], [STOP] (0 to 3), the audio tokens [0] to [7999] (4 to 8003), and 188 text tokens (8004 to 8191): 53 characters (the space, !"$%&'*+,-./<>?_, the digits, a to z) and 135 merges (th, in, ... gonna); tokenizer.ggml.pre = soprano; the beginning, the end and the padding are all token 3 none

The same repository has the language model as F16 (159,632,416 bytes, 2f7f6821..., the checkpoint's own precision) and in smaller quantizations, and files without -soprano-tok in their name whose tokenizer.ggml.pre is gpt-2: those are for codec.cpp and are cut into the wrong tokens by OKLM and by llama.cpp. The catalogue takes Q8_0 because it is the faster here (below) and is heard the same.

The decoder's file is the checkpoint. decoder.pth is a zip archive of a pickle (the tensors' names, shapes and places) and one raw file per tensor. tools/gen_soprano_fixtures.py reads it with Python's standard library and numpy, without PyTorch (its unpickler resolves the tensor constructors of a state dictionary and nothing else):

  • --check-pth decoder.pth codec-F32.gguf: the 81 tensors of the published GGUF file are the checkpoint's, byte for byte, and its metadata is what the checkpoint's shapes give;
  • --convert decoder.pth out.gguf writes that GGUF file from a checkpoint: for decoder.pth the result is codec-F32.gguf byte for byte (cmp), so a decoder trained elsewhere converts the same way;
  • --mmproj decoder.pth out.gguf writes the projector file llama.cpp's llama-tts reads, for the comparison below.

From the text to the prompt

The cleaner. Soprano's package cleans a text before its tokenizer sees it (clean_text() of soprano/utils/text_normalizer.py, adapted there from tortoise-tts); llama.cpp leaves that to the caller. OKLM does it (src/mm/soprano_text.c, oklm_speech_normalize_soprano()), in the package's order:

  1. ASCII: é is e, ß is ss, the typographic quotes and dashes the plain ones, a long dash a dash between spaces;
  2. every line is a sentence: a line that does not end with ., ! or ? gets a period;
  3. numbers: #1 is "number one", 5K "five thousand", 1,234 loses its comma, 12/25/2024 is "twelve dash twenty-five dash twenty twenty-four", (555) 123-4567 its digits one by one, 8:05 "eight oh five", $2.47 "two dollars, forty-seven cents", 3.14 "three point one four", 2 * 3, 6 / 2, 2 + 2 and -5 "times", "over", "plus" and "minus", 21st "twenty-first", mp3 "mp three", and what is left in words: 1001 to 2999 as a year ("nineteen eighty-four", "two thousand nine", "nineteen oh five"), the others as the inflect package writes them ("one million, two hundred thirty-four thousand, five hundred sixty-seven");
  4. https:// spelled, a dash between spaces a comma, a period between two letters "dot", brackets commas;
  5. abbreviations: Dr. "doctor", Mrs. "misess", St. "saint", ...; Hz, MB, API, CPU, Mon, Jan, etc, ... in their own case;
  6. a word with capitals inside is cut before each ("LMDeploy" is "l m deploy"; "TPUs" is "tpu's");
  7. signs: @ "at", & "and", % "percent", : a period, ; a comma, +, \, ~, <, >, =, / in words, _ and * spaces;
  8. lower case; only the letters, the space and !$%&'*,-.? are kept;
  9. one space for several, none before punctuation; .. is ., !! is !, ?! is ?, three periods stay an ellipsis; a letter written three times in a row is written twice.

It is written by hand, each regular expression of the package as a scan of the text, and checked on 386 texts against the package's expressions applied with Python's re (clean_text() of tools/gen_soprano_fixtures.py, numbers spelled by a rewriting of inflect's rules): the two give the same text for all of them, and for 80,000 random texts put together from the pieces the rules look for (digits, signs, brackets, abbreviations, capitals). The package's own cleaner was not run: it needs the inflect and unidecode packages. Where OKLM differs from it, on purpose:

  • characters outside the table of step 1 are dropped; unidecode would write other scripts in Latin letters;
  • £20 is "twenty pounds": the package turns £ into PS before its rule for pounds can see it;
  • a time with seconds reads its minutes as they are ("3:45:10" is "three forty-five ten"); the package writes a Python set there ({'45'});
  • a tab is a space; the package drops it and glues the words;
  • a number above the decillions is read digit by digit, and $,5 is five dollars; inflect and the package raise an error.

It keeps the package's oddities too: "1001" is "ten oh one", "9-5" "nine minus five", "the 1990s" "the nineteen ninety s".

The tokenizer (tokenizer.json: Lowercase and \s+ to a space, then Digits with each digit alone and a Split on \s+|\w+|[^\w\s]+, then BPE without byte encoding, [UNK] for a piece the vocabulary lacks) is the GGUF's tokenizer.ggml.pre = soprano, which OKLM's tokenizer had; the published files do not all carry tokenizer.ggml.unknown_token_id, and OKLM takes [UNK] when they do not (the pinned llama.cpp writes the id -1 for such a piece with these files). On the eight texts of the repository's tokenizer-tests.md, whose counts are the Hugging Face tokenizer's, OKLM gives the same ids from the GGUF file and from tokenizer.json, and the table's count for seven; for the eighth, a line of emoji and signs, OKLM and the pinned llama.cpp both write 29 tokens where the table says 30 (not explained: the Hugging Face tokenizer is not on this machine). A cleaned text has no character outside the vocabulary, so [UNK] is not written in a synthesis.

The prompt is [STOP], [TEXT], the cleaned text's tokens, [START] (3, 1, ..., 2): "hello world, this is a test of speech synthesis." is 36 tokens. The model writes audio tokens until [STOP].

What goes to the decoder is not the tokens but the final normalized hidden state that predicted each of them: the state at [START] for the first, the state at each token written for the next, the one that predicts [STOP] left out (outputs.hidden_states[j][-1] of the package's generate(); llama_get_embeddings_ith() in llama.cpp; oklm_context_params.hidden and oklm_get_hidden() here). oklm_speech_states() gives them for a text.

Sampling. The package's: the most likely token (a temperature of 0.001 under top-p 0.95) with a repetition penalty of 1.2 over the prompt and everything written, at most 512 tokens (32.7 s). OKLM's defaults for this model are those (oklm_speech_options_for(): --temp 0 --top-k 0 --repeat-penalty 1.2; the prompt's tokens count for the penalty, which holds [STOP], the prompt's first token, back a little), where OuteTTS's are temperature 1 and top-k 4. --temp above 0 samples under top-p 0.95. A prompt is also cut at 32 tokens plus three per character of its text (a token is 64 ms, a character of speech about as much): a model that does not stop is stopped there. The package's detector of runs of similar states (its retries) is not there.

Passes. As the package does: every sentence is a prompt of its own, and a sentence whose cleaned text is shorter than 30 characters goes with the one before it (the first one with the one after). The model was trained on such prompts: given two long sentences in one prompt it repeated the end of the second ("... by the seashells by the seashore", heard by both listeners), which it does not when each has its prompt. The sentences are found as for OuteTTS (., ! or ? followed by a space, or a line feed), a sentence of more words than --words (24) is cut after a comma, a semicolon or a colon, then anywhere, and a prompt's text of more than 300 characters is cut again at a space (the package's splitter cuts at 300 too). A prompt costs three tokens beside its text: --prompt-words is not read, and nothing is kept in the cache from one prompt to the next.

The first sound. --first-words N does not cut the text: the sound of the first prompt's first N words, counted as 6 tokens (0.38 s) a word, is handed over as soon as those tokens and the 4 after them are written, then the rest of the sentence when it is done. The decoder is convolutions and a transform of overlapping windows: a sample depends on the states within 4 of its own (the package's RECEPTIVE_FIELD), so the two pieces are the samples of the whole sentence, bit for bit (checked on the three sentences below and in the tests), with no seam to hide.

The decoder

src/mm/vocoder.c reads it beside WavTokenizer's (oklm_vocoder_load() tells them by general.architecture; oklm_vocoder_state_width() is 512 for this one and 0 for a vocoder of codes):

  1. the n states, [512] each, are interpolated linearly to 4 (n - 1) + 1 frames, the first and the last frame being the first and the last state (interpolate(mode='linear', align_corners=True));
  2. sop.decode.embed, a convolution of kernel 1 to 768 channels, and a layer normalization (epsilon 1e-6);
  3. eight ConvNeXt blocks (sop.decode.cnx.N): the depthwise convolution dw (kernel 3), ln, pw1 (768 to 2304), the exact GELU, pw2, the scale gamma, and the input added: the blocks of WavTokenizer's decoder, run by the same code;
  4. sop.decode.fln and sop.decode.head.out: 2050 values per frame, the log-magnitudes of 1025 bins (at most 100 once raised) and their phases;
  5. oklm_istft_center(): the constant and the highest bin of every frame set to zero (the head's own spec[:, 0] = 0; spec[:, -1] = 0), an inverse real transform of 2048 points, a periodic Hann window (the file's sop.decode.istft.window is checked to be one), frames added 512 samples apart, the sum divided by the sum of the squared windows, 1024 samples dropped at each end as torch.istft(center=True) does: (frames - 1) x 512 samples. A state is 2048 samples: 15.6 states a second at 32 kHz.

Comparison with llama.cpp and with a double-precision decoder

The reference. The pinned llama.cpp (build/llama.cpp, 8642541b) runs Soprano in llama-tts. That one tool was built from the pinned sources for the CPU in a directory of its own (cmake -DGGML_NATIVE=ON -DLLAMA_CURL=OFF, the tree used as a subdirectory so that nothing is written in it), with a copy of tools/tts/tts.cpp that also writes the text's ids, the tokens sampled, the states handed to its decoder and the samples as floats. Its files: the same language model file as OKLM, and the projector file tools/gen_soprano_fixtures.py --mmproj writes from decoder.pth (what llama.cpp's converter writes with PyTorch: the same F32 values, the depthwise kernels in half precision because ggml's convolution reads no other). Both programs ran the F16 language model with the most likely token and no repeat penalty, four threads, on the cleaned texts "hello world, this is a test of speech synthesis.", "the quick brown fox jumps over the lazy dog." and "it costs forty-two dollars and arrives on monday morning.". The third implementation is the numpy decoder of tools/gen_soprano_fixtures.py --run, in double precision, written from the package's soprano/vocos/*.py.

Compared Sentence 1 Sentence 2 Sentence 3
The text's token ids, OKLM and llama.cpp the same 33 the same 34 the same 38
The tokens written the same 48 the same 42 the same 52
Hidden states, OKLM against llama.cpp (root mean square 1) 63.9 dB, largest difference 0.007 65.0 dB, 0.004 65.2 dB, 0.006
Rows (log-magnitude, phase): OKLM against numpy, the same states 134.8 dB 126.3 dB 124.8 dB
Samples: OKLM against numpy, the same states 122.0 dB, largest difference 1.0e-6 110.2 dB, 4.7e-6 107.4 dB, 1.1e-5
Samples: the synthesis with first_words (two pieces) against one call of the decoder identical identical identical
Samples: llama.cpp against numpy on llama.cpp's states, the two bins kept and the depthwise kernels in half precision 57.3 dB 43.6 dB 39.6 dB
Samples: numpy with the two bins kept against numpy without 38.9 dB 39.5 dB 39.9 dB
Samples: numpy on OKLM's states against numpy on llama.cpp's 46.4 dB 30.1 dB 23.6 dB
Samples: OKLM against llama.cpp, each from its own states 37.8 dB 27.9 dB 21.5 dB

The samples are in [-1, 1] with a root mean square of 0.08. Read together: the language model is llama.cpp's to the token (142 tokens of 142) and to 64 dB on the states; OKLM's decoder is the definition's to 107 dB or better. llama.cpp's decoder is the same network (mean difference 4e-5 to 1.4e-4 of a sample, a few frames further off) with one difference of definition: it keeps the constant and the highest frequency bin, which the model's head sets to zero; that alone is the 39 dB of the eighth row. The last two rows are not a decoder's error: the decoder turns phases into samples, and states that differ at 64 dB come out 24 to 46 dB apart whoever computes them (ninth row, one decoder on both programs' states).

The Q8_0 file the catalogue installs: its states are 35 to 37 dB from the F16 file's in both programs (the quantization, not the program), the two programs write the same first 17, 42 and 28 tokens of the three sentences and then choose differently between near-equal tokens, as they do for OuteTTS.

What was not compared: the package itself (PyTorch is not on this machine). The decoder's reference is its definition rewritten in numpy; the language model's is llama.cpp; the cleaner's is the package's expressions under Python's re.

Heard

Six sentences spoken one by one by oklm speak -m soprano-1.1-80m (the defaults), transcribed by oklm transcribe:

Sentence Heard by Whisper base Heard by Whisper small
Hello world, this is a test of speech synthesis. the same, 9 of 9 the same, 9 of 9
The quick brown fox jumps over the lazy dog. the same, 9 of 9 the same, 9 of 9
It costs 42 dollars and arrives on Monday morning. "It costs $42 and arrives on Monday morning.", 10 of 10 the same words, 10 of 10
Please open the window before you leave the room. the same, 9 of 9 the same, 9 of 9
The model writes audio codes and the vocoder turns them into sound. "... and the VACOTER turns them into sound.", 11 of 12 "... and the Vockoder turns them into sound.", 11 of 12
She sells sea shells by the sea shore. "She sells seashells by the seashore.", 4 of 8 the same, 4 of 8
Words right 52 of 57 (91.2%); 56 with "seashells" and "seashore" counted right 52 of 57; 56 the same way

Every word is heard but "vocoder", which neither Whisper knows; OuteTTS 0.2 on the same sentences is heard 53 and 50 of 57 (above), with a wrong ending and a wrong start that Soprano does not have. The six sentences as one text (17.2 s of sound, six prompts) are heard the same way by both: every word but "vocoder" ("Vacoter", "Vockoder"), and with --temp 0.3 --seed 7 too. The 64-word text of the table above comes back from Whisper small as "Speech synthesis turns a written text into sound. The language model reads the words and writes audio code 75 for each second. The Valkoder then turns the codes into a waveform at 24 kHz. A long text is cut into passes of a few sentences and each pass is spoken on its own. This paragraph has five sentences and takes several passes.": "codes," and "vocoder" misheard. And "Dr. Smith paid $2.47 on 3/4/2025 at 8:05 for the 2nd API key, i.e. about 50% of 1,200 MB." (cleaned to "doctor smith paid two dollars, forty-seven cents on three dash four dash twenty twenty-five at eight oh five for the second a p i key, i dot e. about fifty percent of twelve hundred megabyte.") as "Dr. Smith paid $2.47 on 3-4-2025-805 for the second API key I.E. about 50% of 1200 megabytes."

Measured against OuteTTS 0.2

The text of five sentences and 64 words of the table above, measured on 2026-11-15 with OuteTTS's sentences passes of their own: on the CPU, OKLM_THREADS=4 on the Core i7-11700K (8 cores, 16 threads) while another job kept the load average at 4.1 to 5.1; on the RX 7900 XT, the language model and the decoder's network on the device. Three runs of each, one after the other:

Backend Prompts, their tokens Tokens written Sound First sound after The whole text after Sound per second
OuteTTS 0.2 (Q8_0), the defaults CPU 2, 1913 1828 20.86 s 6.6 s 40.9 to 41.0 s 0.51
OpenCL 2, 1913 1852 21.18 s 0.86 to 0.87 s 5.76 to 5.80 s 3.65 to 3.67
Vulkan 2, 1913 1820 20.75 s 0.65 to 0.66 s 4.31 to 4.34 s 4.78 to 4.82
OuteTTS 0.2 with --first-words 5 CPU 2, 1913 1828 20.86 s 4.6 s 40.8 to 40.9 s 0.51
OpenCL 2, 1913 1852 21.18 s 0.57 to 0.58 s 5.75 to 5.78 s 3.66 to 3.68
Vulkan 2, 1913 1820 20.75 s 0.44 s 4.34 to 4.35 s 4.77 to 4.78
Soprano 1.1 (Q8_0, the catalogue's), the defaults CPU 5, 275 308 19.39 s 0.20 to 0.21 s 1.48 to 1.54 s 12.6 to 13.1
OpenCL 5, 275 312 19.65 s 0.06 to 0.07 s 0.50 to 0.51 s 38.8 to 39.6
Vulkan 5, 275 313 19.71 s 0.06 s 0.42 to 0.45 s 44.2 to 47.1
Soprano 1.1 (Q8_0) with --first-words 5 CPU 5, 275 308 19.39 s 0.17 s 1.49 to 1.62 s 12.0 to 13.0
OpenCL 5, 275 312 19.65 s 0.05 s 0.50 to 0.51 s 38.4 to 39.5
Vulkan 5, 275 313 19.71 s 0.05 s 0.42 to 0.44 s 45.3 to 46.6
Soprano 1.1 (F16), the defaults: an earlier series CPU, load 15 to 18 5, 276 319 20.10 s 1.71 s, 1.58 s, 1.25 s 9.5 s, 11.4 s, 7.9 s 2.12, 1.76, 2.56
Soprano 1.1 (F16) with --first-words 5: an earlier series CPU, load 15 to 18 5, 276 319 20.10 s 1.28 s, 1.08 s, 1.34 s 8.2 s, 11.2 s, 8.1 s 2.44, 1.80, 2.47

(The F16 file is not on the machine of 2026-11-15: its two rows are those of the series before, made under six other jobs, where the Q8_0 file gave 2.7 to 4.0 seconds of sound a second and OuteTTS 0.2 0.08 to 0.23. With nothing else running OuteTTS 0.2 made 0.43 second of sound a second at best in the runs of Limits.)

Soprano makes 12.6 to 13.1 seconds of sound a second on the CPU where OuteTTS 0.2 makes 0.51 under the same load: 25 times faster, and faster than it is heard with room to spare. Its first sound comes after 0.2 s without asking for anything (its first prompt is the first sentence, eight words here), 0.17 s with --first-words 5, against 6.6 s, or 4.6 s with --first-words 5, for OuteTTS. On a device both are faster than they are heard: OuteTTS 3.7 seconds of sound a second on OpenCL and 4.8 on Vulkan, its first sound after 0.9 and 0.7 s (0.6 and 0.4 with --first-words 5); Soprano 39 and 44 to 47, its first sound after 0.06 s. Where the time goes for the Q8_0 file on the CPU: the five prompts 0.07 s, the 308 tokens 0.86 to 0.97 s (320 to 360 a second), the decoder and its transform 0.54 to 0.57 s. A sentence alone is spoken at 2.5 to 4.6 seconds of sound a second (the six of the table above, under the earlier load).

A second series of the Q8_0 file alone, later (load 19 to 21): the text after 4.3 s, 4.4 s and 3.9 s (4.5 to 5.0 seconds of sound a second), its first sound after 0.76 s, 0.46 s and 0.50 s; with --first-words 5 after 3.7 s and 3.8 s, the first sound after 0.39 s and 0.45 s. Peak memory is 397 MiB for this text, where OuteTTS takes 1.65 GiB.

Which model the examples name

OKLM has no default speech model: oklm speak and oklm serve --speak-model take the model they are given. The examples name one model, outetts-0.2-500m: those of README.md, of this page, of server.md and of gui.md, and the hints the chat window and the dashboard's page give when nothing speaks. Another model is named only on a line that is about it (OuteTTS 0.3, Soprano), and wherever an example starts a server or a window with OuteTTS, a line beside it does the same with Soprano. Until 2026-11-15 the Speech section of server.md and the README's device example named outetts-0.3-500m.

Why OuteTTS 0.2 and not Soprano, from what is weighed below: the examples about voices (--speak-voice, --speak-voices, the Voice chooser, the route's voice) only run with a model that takes speaker files; the transcripts and figures of gui.md and server.md were made with it and stay true as they are; a client written for OpenAI's route assumes 24 kHz; and since 2026-11-14 its language model runs on a device (3.7 seconds of sound a second on the RX 7900 XT with --backend opencl, the 64-word text above), which takes away most of what speed argued for where there is a GPU. Soprano is the one to take on a CPU alone, or where a licence must allow commercial use: the lines beside the examples say so.

  • For: 2.7 to 4.0 seconds of sound a second on four threads of a loaded CPU against 0.08 to 0.23, and a first sound in about a second against half a minute (the table above); a licence that allows commercial use (Apache-2.0; OuteTTS 0.2 is non-commercial); 197 MiB against 636; punctuation, numbers, dates and abbreviations read; heard at least as well.
  • Against: one voice and no speaker files (OuteTTS takes any voice made by its package); another voice than the one users of OuteTTS know; 32 kHz where a client written for OuteTTS may assume 24 kHz; a catalogue file converted by a third party (its decoder checked byte for byte against the author's checkpoint, its language model against llama.cpp, not against the author's weights).
  • What switching would touch: the examples of README.md, docs/speech.md, docs/server.md and docs/gui.md and the quick start (--speak-model soprano-1.1-80m); the hints of the chat window and of the dashboard's page, and the tests that read them; the chat window's Voice chooser, which would list one voice (the window plays at the rate of the model that speaks, and picks the installed speech model by itself); nothing else in the dashboard, whose player reads the rate from X-Sample-Rate and speak_stats.sample_rate; the tests that name outetts-0.2-500m as the installed model (tests/server/test_server.c, tests/mm/test_speech.c), which stay as tests of OuteTTS.

Limits of Soprano

  • One fixed voice, English. No speaker file, no cloning (the model has no reference in its prompt); the package's Soprano-Encoder repository is for training.
  • The package itself was not run (see above): three implementations agree, none of them is PyTorch's.
  • A sentence at a time. The model's prosody is a sentence's; between two sentences there is the pause the model ends and begins with, nothing added.
  • The cleaner is English and literal: "9-5" is "nine minus five", years are guessed from the number alone, other scripts are dropped.
  • Mispronounced words: the model card gives 1,000 hours of training speech; rare words ("vocoder") come out wrong, as with OuteTTS.
  • The decoder's inverse transform on the CPU. --backend opencl|vulkan puts the language model and the decoder's network on a device (on a device).
  • The decoder is about two fifths of the time on this CPU (F32 products of 768 x 2304, 62 frames a second): it was not tuned.
  • Soprano 1.0 (ekwek/Soprano-80M, a decoder of 58 MB with other sizes) was not tried: its checkpoint would go through --convert, whose metadata is read from the tensors' shapes.

The library

include/oklm/mm/speech.h:

int oklm_speech_open(const char* model, const char* vocoder, const oklm_speech_params* p,
                     oklm_speech** out, char* err, size_t err_size);
int oklm_speech_synthesize(oklm_speech* s, const char* text, const oklm_speech_options* o,
                           float** samples, size_t* n);
int oklm_speech_synthesize_chunks(oklm_speech* s, const char* text, const oklm_speech_options* o,
                                  oklm_speech_chunk_fn fn, void* user);
int oklm_speech_set_voice(oklm_speech* s, const char* voice, char* err, size_t err_size);
int oklm_speech_voice_check(const char* path);
void oklm_speech_last_stats(const oklm_speech* s, oklm_speech_stats* out);
int oklm_speech_stretch(const float* samples, size_t n, int rate, float speed, float** out, size_t* n_out);
int oklm_speech_rate(const oklm_speech* s);
void oklm_speech_free(oklm_speech** s);
int oklm_speech_wav(const float* samples, size_t n, int rate, unsigned char** wav, size_t* size);
void oklm_speech_pcm16(const float* samples, size_t n, int16_t* out);

int oklm_speech_format(const oklm_speech* s);                 /* OKLM_SPEECH_OUTETTS_0_2, _0_3 or OKLM_SPEECH_SOPRANO */
oklm_speech_options oklm_speech_options_for(int format);      /* the sampling a model is meant to run with */
int oklm_speech_probe(const char* model, const char* vocoder, int* format, int* rate);
int oklm_speech_normalize_soprano(const char* text, char** out);
int oklm_speech_states(oklm_speech* s, const char* text, const oklm_speech_options* o, float** states, size_t* n, int32_t** tokens);
int oklm_vocoder_decode_states(oklm_vocoder* v, const float* states, size_t n, float** samples, size_t* n_samples);

oklm_speech_params holds the backend of the language model ("cpu", "opencl", "vulkan", "auto"), a speaker file and the context; oklm_speech_options the sampling settings, the guide, the words per pass (max_words), per prompt (prompt_words) and of the first pass (first_words), the speed, and a stop flag read between tokens. oklm_speech_set_voice() gives an open synthesizer another speaker file, or the built-in voice again; oklm_speech_voice_check() says whether a file is a speaker file. oklm_speech_last_stats() gives what the last synthesis cost (oklm_speech_stats: prompts, passes, prompt_tokens, cached_tokens, generated_tokens, prompt_seconds, generate_seconds, vocoder_seconds, first_seconds, seconds, audio_seconds). The samples are mono floats in about [-1, 1] at oklm_speech_rate() (24000); oklm_speech_wav() makes a 16-bit WAV file in memory and oklm_speech_wav_write() writes one; include/oklm/mm/flac.h has oklm_flac_encode(), oklm_flac_writer_new() / _add() / _free() and oklm_flac_decode(). A synthesizer runs one synthesis at a time: callers that share one serialize their calls. The pieces are public too: oklm_speech_normalize(), oklm_speech_split(), oklm_speech_split_first(), oklm_speech_prompt_text(), oklm_speech_prompt_tokens(), oklm_speech_codes() (the language model's half) and the vocoder alone (oklm_vocoder_load(), oklm_vocoder_spectrum(), oklm_vocoder_decode(), oklm_istft()).

A client written for OuteTTS runs Soprano unchanged when it reads the rate from oklm_speech_rate() (32000) instead of OKLM_SPEECH_RATE, takes its options from oklm_speech_options_for(oklm_speech_format(s)) (or passes none) instead of oklm_speech_default_options(), which are OuteTTS's, and offers no speaker file when the format is OKLM_SPEECH_SOPRANO. oklm_speech_probe() gives the format and the rate from the files' headers, before anything is loaded (what the server does). oklm_speech_codes() has no meaning for Soprano (OKLM_ERR_UNSUPPORTED): its counterpart is oklm_speech_states(), with oklm_vocoder_spectrum_states(), oklm_vocoder_decode_states() and oklm_istft_center() for the decoder alone, and oklm_vocoder_probe() to tell the two kinds of vocoder file apart.

Tests

tests/mm/test_speech.c, with fixtures written by tools/gen_speech_fixtures.py (numpy only): the normalization of eleven texts and the prompts of three with a small speaker file, against the script's own implementation; the cut into passes and its shorter first pass; the change of speed on a tone (length, pitch, level, the first samples); FLAC files of ten lengths and five kinds of sound read back sample for sample, a stream written in pieces, damaged files refused; the inverse transform against numpy's irfft for sizes 40, 28 and 1280 (factors 2, 5 and 7); a tiny random vocoder in the published file's layout (the same tensor names, shapes and types) against the script's double-precision network, rows and samples; WAV files read back; the refusals (a backend the machine does not have, a file that is not a vocoder, a language model that is not a speech model, codes outside the codebook). When models/outetts-0.2-500m is there: the prompt's ids against llama.cpp's (their count and hash), the vocoder's rows and samples on 32 codes against the numpy implementation run on the published file, and a word spoken end to end. The vocoder's network on each device backend found against the CPU's samples, on the tiny file (test_vocoder_devices) and on the published one with the codes of "Hello world" (test_published_devices): the ratio of the sound to the difference, a shorter then a longer call, a second call, the vocoder back on the CPU, a device with no room and OKLM_MM_DEVICE_ENDS=0; and, on the tests' counting device with a bound (tests/mm/count_backend.h, the CPU's kernels behind a device's interface), a device with room for the network's weights and not for a call's activations: the call is computed on the CPU, to its bits, the network staying there. The passes of a text under one prompt and under one each (their counts, the tokens kept in the cache, the same sound), a voice changed and one refused are in tests/server/test_server.c (test_speech_route), on the tiny speech model that test builds, with the route's flac, speed, voice and GET /v1/audio/voices.

Soprano: tests/mm/test_soprano.c, with fixtures written by tools/gen_soprano_fixtures.py (numpy only, in tests/fixtures/mm/tiny-soprano/): the cleaner on 386 texts against the script's; the inverse transform with "center" padding against numpy's irfft for sizes 16, 40 and 2048; a tiny random decoder in the published file's layout against the script's double-precision network, rows and samples, and its sound in pieces against its sound in one call; a tiny language model with Soprano's kind of vocabulary that writes twelve tokens after any prompt, on which the whole synthesis runs: the prompt's ids, the states, a prompt per sentence, the short sentences put together, a long one cut, the first sound early with the same samples, the speed, a stop, and the refusals (a speaker file, codes, a language model and a vocoder that do not go together). When models/soprano-1.1-80m is there: the prompt's ids and the first tokens written against llama.cpp's, the decoder on the states (and on each device backend against the CPU's samples, as the tiny decoder is by test_decoder_devices), a sentence spoken with its first sound early. test_speech_route_soprano of tests/server/test_server.c runs the route on the tiny files: 32 kHz in the headers and the WAV file, the synthesizer's own samples as a file and streamed, the one voice.

Other models

What running each would take, from their model cards and repositories; no code here.

Soprano is no longer here: it runs (Soprano).

Kokoro (hexgrad/Kokoro-82M; Apache-2.0; 8 languages, 54 voices, 24 kHz). 82 M parameters as a PyTorch checkpoint (kokoro-v1_0.pth, 327 MB) and one tensor file per voice; built on StyleTTS 2 with an ISTFTNet vocoder. It reads phonemes, not text: its package uses the misaki grapheme-to-phoneme library with eSpeak NG behind it. It takes a converter from the checkpoint, kernels OKLM does not have (bidirectional LSTMs for durations and prosody, transposed convolutions, adaptive instance normalization), and above all a grapheme-to-phoneme front end per language. No GGUF and no llama.cpp support: the reference would be its Python package.

Piper (rhasspy/piper-voices; the voices' licences vary with their datasets, the repository's card says MIT; more than 30 languages). One VITS model per voice, published as ONNX (.onnx and a .onnx.json with the phoneme table), 16 or 22.05 kHz, fast on a CPU. It also reads phonemes, made by eSpeak NG (GPL-3.0), which OKLM would have to call as a program or replace. It takes an ONNX reader or a converter, and VITS's blocks: a transformer text encoder with relative positions, a duration predictor, normalizing flows, a HiFi-GAN decoder (transposed convolutions). The reference would be ONNX Runtime.

OuteTTS 1.0 (Llama-OuteTTS-1.0-1B, CC-BY-NC-SA-4.0) changed the audio codec (DAC) and is another vocoder. OuteTTS 0.3 also exists as a 1B model (OuteAI/OuteTTS-0.3-1B-GGUF, CC-BY-NC-SA-4.0 with its Llama base), which the same code would open and the catalogue does not list.

On a device

oklm speak --backend opencl|vulkan|auto (oklm_speech_params::backend) creates the language model's context on a device: the prompt, the voice kept in the cache and every token written are computed there as for any model of its architecture, the logits (Soprano: the hidden states too) read back a token. The vocoder's network (WavTokenizer's decoder, Soprano's decoder) computes there too since 2026-11-14 (the vocoder on a device); the inverse transform stays on the CPU.

Where the time of a sentence of 7 seconds goes (RX 7900 XT, measured again on 2026-11-15, load average 1.9 to 3.7: a sentence of 23 words, --seed 1, two runs each, seconds; the prompt, the tokens written, the vocoder, then the whole command's synthesis; in the two device columns the vocoder's network is on the device, its group normalizations and attention block included, and the inverse transform is on the worker team in all three):

Model CPU OpenCL Vulkan
OuteTTS 0.3 500M Q8_0 prompt of 907 tokens 0.56 to 0.59 0.05 0.05
603 to 648 tokens written 12.86 1.77 to 1.79 1.39
vocoder 0.22 0.02 0.02
the sentence 13.67 to 13.69 1.90 to 1.92 1.48
Soprano 1.1 80M Q8_0 109 to 113 tokens written 0.30 to 0.31 0.18 0.13
decoder 0.11 0.02 0.02 to 0.03
the sentence 0.43 to 0.44 0.21 to 0.22 0.17

On the CPU the language model is 94 % of OuteTTS's time and the vocoder 1.6 %; on a device the sentence is 3.5 to 3.6 s of sound a second of work on OpenCL and 4.7 on Vulkan against 0.54 on the CPU, and the vocoder is 1 % of it. While the vocoder stayed on the CPU (the same day's first table: 0.31 to 0.33 s of a sentence of 1.83 to 2.25 s) it was 15 to 17 % of the sentence. Soprano speaks 15.6 to 15.9 s of sound a second on the CPU, 32.9 to 33.5 on OpenCL and 42.4 to 43.1 on Vulkan; its decoder is a quarter of its time on the CPU and a tenth on a device.

The sound is not the CPU's sample for sample: a device's products sum in another order, the codes of a sound are many near ties, and a code that differs changes every code after it. Measured (test_published_devices in tests/mm/test_speech.c): on the prompt of "Hello world" and the 96 tokens the CPU then writes greedily, the device's best token is the CPU's in 92 of 96 steps for OuteTTS 0.2, 94 or 95 for OuteTTS 0.3 and 95 for Soprano, and where they differ the CPU's margin between the two tokens is 0.063 at most; the codes of a word written greedily are the CPU's for the first 42 (OuteTTS 0.2) and 6 or 8 (OuteTTS 0.3), of the same count. Soprano's tiny model gives the CPU's hidden states on both backends (test_tiny_devices in tests/mm/test_soprano.c).

Limits: a backend the machine does not have is refused.

The vocoder on a device

oklm_vocoder_use_backend() copies a vocoder's network to a device (oklm speak --backend and a server's --speak-backend do it when the language model has layers there), and oklm_vocoder_spectrum() and oklm_vocoder_spectrum_states() then compute a call's rows there, one pass on the device a call. Where a call's time went on the CPU (RX 7900 XT host, 6.4 seconds of sound in two passes, milliseconds): WavTokenizer's network 97 and 150, its inverse transform 20 and 32; Soprano's network 39 and 53, its inverse transform 26 and 37.

The network is products, norms and activations the backends have, and convolutions, which they have not as an op. No op was added (src/mm/vocoder.c, vd_forward()):

  • A convolution of kernel k is k products: the kernel is copied to the device as one matrix a position, and each product reads a view of the rows that position reads, the first written, the others added (OKLM_OP_MATMUL accumulating), with the activations at F32's precision (oklm_matmul_wide()). An activation a convolution reads is kept between four rows of zeros at each end: what the kernel reads past the frames (kernels up to nine positions; a longer one leaves the vocoder on the CPU).
  • The depthwise convolution of a ConvNeXt block is, for each position of its kernel, the rows that position reads times that position's weights, a vector, added to the bias then to the sum so far: the CPU's order.
  • Group normalization (WavTokenizer's "posnet" stack, ten of them a call) reduces over a group's channels and every frame. A frame holds a group's channels side by side, so a view of the stream [channels a group, frames, groups] names them, and a copy through it lays each group out as one row; a layer normalization of those rows is the group's mean and variance; a copy through the same view of the destination puts the values back, and the scale and the shift by channel are a product and a sum (five launches, six with the swish).
  • The attention block of that stack has one head of 768 values, wider than Vulkan's attention shaders take (256). It is computed as products on both backends: the keys read as a matrix times the queries are the scores, their softmax the weights, and the values copied by channel (a transposed view) times the weights the output. The products run over the frames rounded up to 32, the keys and the values past the frames rows of zeros.
  • The lookup of the codes, Soprano's interpolation of the states and the inverse transform stay in host memory, the last on the worker team (above).

Measured on 2026-11-14 (RX 7900 XT, --seed 1, the sentence of 23 words of the test, 6.3 to 6.5 seconds of sound, two runs each; the vocoder's time, then the whole synthesis, with the language model on the device in every column):

Model Vocoder on the CPU, as before The transform on the team The network on the device
OuteTTS 0.3 500M Q8_0 with WavTokenizer Vulkan 0.30 s, 1.67 s 0.25 to 0.26 s, 1.61 to 1.62 s 0.06 s, 1.42 s
OpenCL not run 0.26 s, 2.07 to 2.10 s 0.05 s, 1.89 to 1.90 s
Soprano 1.1 80M Q8_0 Vulkan 0.16 s, 0.28 s 0.10 s, 0.23 s 0.02 s, 0.14 s
OpenCL not run 0.10 s, 0.25 s 0.02 s, 0.16 to 0.17 s

WavTokenizer's network takes 23 and 29 ms on Vulkan and 17 and 25 ms on OpenCL for the two passes (97 and 150 on the CPU), Soprano's 4.8 and 5.2 ms and 3.5 and 3.9 ms (39 and 53). Of WavTokenizer's, the "posnet" stack with its group normalizations on the CPU was 16 and 21 ms on Vulkan and 10 and 16 ms on OpenCL. With the group normalizations and the attention block on the device (2026-11-14, a sentence of 23 words in one pass of 507 to 527 frames, two runs each) the network takes 14.2 to 14.3 ms on Vulkan where it took 43.9 to 44.6 ms, and 13.5 to 13.7 ms on OpenCL where it took 32.9 to 33.8 ms; the 16-bit files differ from those of the normalizations on the CPU by one step in 207 and 272 of 168 640 and 162 240 samples (99 and 97 dB). For that sentence the command's vocoder time went from 0.05 s to 0.02 s on Vulkan and from 0.04 s to 0.02 s on OpenCL, and the sentence from 1.51 s to 1.47 to 1.48 s and from 1.90 to 1.95 s to 1.88 to 1.92 s; the table above is from before.

The sound against the CPU's vocoder on the same codes or states: the 16-bit files of the sentence above differ by one step at most, in 263 to 1 267 of 153 280 to 200 704 samples (90 to 97 dB). In floats (test_vocoder_devices, test_published_devices and test_decoder_devices): WavTokenizer on the codes of "Hello world" within 1.2e-6 on OpenCL and 5.9e-6 on Vulkan (125 and 110 dB), Soprano's decoder on a sentence's states within 3.5e-6 and 3.3e-6 (112 and 113 dB), the tiny files 117 to 132 dB; the tests ask for 80 dB, a second call to give the first's bits, and the vocoder back on the CPU the CPU's. The convolutions' and the depthwise form's ops, the group normalization's and the one head's are checked against the CPU's arithmetic by test_conv_views (tests/core/test_opencl.c, tests/core/test_vulkan_gemm.c; RX 7900 XT and Intel UHD 750). The language model's tokens are what they were: its figures above are measured again by the same test with the vocoder on the device (92 of 96, 94 or 95, 95).

Limits: the device holds the network's weights (WavTokenizer's file is its size) and a call's activations, which a longer call asks for when it comes (oklm_vocoder_device_bytes()); a device without room for the weights leaves the vocoder on the CPU with a notice, one without room for a call's activations leaves that call to the CPU (WavTokenizer's attention adds a row of the frames a frame to them: 17 MiB at the 2 048 frames a server counts, 1 MiB for a sentence of 7 s). A call reads nothing back but its spectrum. OKLM_MM_DEVICE_ENDS=0 keeps the network on the CPU (build.md). On the Intel UHD 750 the tiny files ran on both backends (118 to 131 dB), no published vocoder.

A server's synthesizer takes a device with --speak-backend (cpu by default): each token of its language model is a pass on the device, and each call of its vocoder, that takes its turn with the scheduler's steps (server.md); a request streaming a completion meanwhile saw no pause (its tokens at most 0.02 to 0.03 s apart, 0.007 to 0.011 s usually), and a sentence of 2.9 s was spoken in 0.76 s (OuteTTS 0.2, Vulkan) and 0.90 s (OpenCL), one of 2.6 s in 0.13 to 0.16 s by Soprano on either (measured with the vocoder on the CPU). With the vocoder's network on the device, a pass of 4 to 29 ms a call that lets a waiting pass in after its first stack, the stream's tokens were at most 0.03 s apart for OuteTTS 0.2 and 0.02 s for Soprano on both backends (0.04 to 0.07 s and 0.02 s before the passes ran in the order they ask; 2026-11-14). The server leaves the synthesizer its room on the device before it sizes its cache (oklm_speech_device_bytes(): the language model's weights, cache and activations, the vocoder's network and the activations of its longest call): 903 and 259 MiB counted for OuteTTS 0.2 with WavTokenizer, 122 and 226 MiB for Soprano, where the device gave 1 070 to 1 078 and 270 to 274 MiB after a first sentence (2026-11-15; 903 and 243, 122 and 244 counted before WavTokenizer's attention had its rows there and the activations were counted as twelve) (server.md).