GGUF
Byte-level layout of GGUF files as OKLM reads and writes them, the
validation rules the reader applies, and how sharded models are handled.
API: include/oklm/io/gguf.h.
Status: implemented (read v2/v3, write v3, sharded sets).
Specification: ggml docs/gguf.md (https://github.com/ggml-org/ggml/blob/master/docs/gguf.md).
Writer layout checked against gguf-py 0.19.0 (the Python
package published from llama.cpp) and against files written by
llama.cpp's C++ quantizer.
Layout
All integers are little-endian. A GGUF string is a
uint64 byte length followed by that many UTF-8 bytes,
without terminator.
| Offset | Field | Type |
|---|---|---|
| 0 | magic GGUF (0x46554747 as uint32) |
uint32 |
| 4 | version (2 or 3) | uint32 |
| 8 | tensor count | uint64 |
| 16 | metadata entry count | uint64 |
| 24 | metadata entries | see below |
| ... | tensor infos | see below |
| ... | zero padding to general.alignment |
|
| data | tensor data, each tensor padded to the alignment |
Metadata entry: key (string), value type (uint32), value. Value types:
| Id | Type | Encoding |
|---|---|---|
| 0 / 1 | uint8 / int8 | 1 byte |
| 2 / 3 | uint16 / int16 | 2 bytes |
| 4 / 5 | uint32 / int32 | 4 bytes |
| 6 | float32 | 4 bytes |
| 7 | bool | 1 byte, 0 or 1 |
| 8 | string | uint64 length + bytes |
| 9 | array | uint32 element type, uint64 count, elements (arrays may nest) |
| 10 / 11 | uint64 / int64 | 8 bytes |
| 12 | float64 | 8 bytes |
Tensor info: name (string), uint32 n_dims,
uint64 ne[n_dims] (innermost first, the same order as
oklm_tensor.ne), uint32 type (ggml type id,
see quant.md), uint64 offset
relative to the start of the data section. The data section starts at
the end of the tensor infos rounded up to general.alignment
(default 32).
Worked example
tests/fixtures/gguf/kitchen.gguf (written by gguf-py)
starts with 47 47 55 46 (magic), 03 00 00 00
(version 3), 05 00.. (5 tensors), 14 00.. (20
entries), then the first entry: key length 14 00.. and
general.architecture, type 08 00 00 00
(string), length 04 00.., test. Its first
tensor, f32 of shape 5 x 3, has offset 0 and its second,
f16, offset 64: 60 bytes of F32 data padded to 64.
Reader validation
The reader (oklm_gguf_open(),
oklm_gguf_open_memory()) maps the file and rejects, with
OKLM_ERR_FORMAT and a logged reason:
- a bad magic, an unknown version (1 and big-endian files give
OKLM_ERR_UNSUPPORTEDwith a specific message); - counts that cannot fit in the bytes left (every entry needs at least 12 bytes, every tensor info 32), checked before allocating anything;
- any string, array or value running past the end of the file; bools other than 0 or 1; unknown value types; arrays nested deeper than 8;
- empty keys, keys with NUL bytes, duplicate keys;
general.alignmentthat is not a power-of-two uint32;- tensor names empty, longer than 63 bytes, with NUL bytes, or duplicated;
- 0 or more than 4 dimensions, retired or unknown types, rows that are not a whole number of blocks, sizes that overflow;
- tensor offsets not aligned, data outside the file, or two tensors whose data overlap.
The reader is more lenient than ggml in one respect: it does not require tensors to be stored in the order of their infos, only aligned, inside the file and non-overlapping.
Values are exposed as parsed oklm_gguf_values. Arrays of
scalars point at the raw elements in the mapping (read them with
oklm_gguf_arr_int() / oklm_gguf_arr_float(),
which handle unaligned little-endian data); arrays of strings and arrays
of arrays have parsed items. Typed getters
(oklm_gguf_get_int(), _float(),
_bool(), _str(), _arr()) accept
any integer width, since files disagree on uint32 vs int32 for the same
key.
The parser is fuzzed (make fuzz-run,
fuzz/fuzz_gguf.c), every truncation of a fixture is tested,
and 20 000 random header corruptions run under ASan in
tests/io/test_gguf.c.
Writer
oklm_gguf_writer reproduces gguf-py's GGUFWriter layout
exactly:
- header (version 3);
- metadata in insertion order (setting an existing key replaces its value in place);
- tensor infos in insertion order, offsets accumulated with each tensor's size padded to the alignment;
- zero padding to the alignment;
- each tensor's data followed by zero padding to the alignment, the last tensor included.
oklm_gguf_writer_set_alignment() also writes
general.alignment, and copying a
general.alignment key from a reader changes the data
alignment, so a file copied key by key keeps its layout. Tensors can be
given as borrowed memory or as a producer callback called during the
write, which keeps conversions at one tensor in memory. Files are
written to path.tmp and renamed.
A file read and written back with every key and tensor copied in order is byte-identical: checked on gguf-py fixtures (all value types, nested arrays, custom alignment, shards) and on the published SmolLM2 Q4_K_M, Q8_0 and F16 files written by llama.cpp.
Sharded models
llama.cpp's gguf-split and gguf-py write shards named
<prefix>-00001-of-0000N.gguf, each with
split.no (uint16, 0-based), split.count
(uint16) and split.tensors.count (int32); metadata other
than split.* is only in shard 0.
oklm_gguf_set_open() opens the model from any shard: it
derives the sibling names, checks every shard's
split.no/split.count, that tensor names are
unique across shards and that their total equals
split.tensors.count, and presents one tensor list. A
single-file model is a set of one.
oklm_gguf_split() / oklm split write shards
as gguf-split --split does: by maximum size (each tensor
counted rounded up to 32 bytes) or by tensor count; shard 0 gets every
key of the input followed by split.no,
split.tensors.count and split.count, the other
shards only those three keys. oklm_gguf_merge() /
oklm merge write one file from the shards, keeping shard
0's keys with split.count set to 0, as
gguf-split --merge. Both outputs are identical byte for
byte to llama.cpp's (tools/check_split.sh, part of
make test-public).