OKLMdocumentation v1.0.3
On this page

Security training design (defensive)

How the "Security training" step of the roadmap's "Default models" is scoped and built. Decided defensive (2026-11-05): the fine-tune teaches the model to find and explain vulnerabilities in code — secure code review, not offence. Defensive review never triggers refusals, so it trains on the normal Qwen3-8B, not the abliterated base; abliteration stays a separate capability.

Status: in progress. The corpus, the data and eval pipelines and two LoRA runs are done (2026-11-07): the first did not change the model's review, the second learned its corrections but traded detection for fewer false positives ("Results" below); the next run is planned there.

Scope

The vulnerability classes are the security-review skill's taxonomy (src/harness/skills.c): injection (SQL, command, template), authentication and authorization flaws, hard-coded secrets, unsafe deserialization, path traversal, SSRF, cryptography misuse, memory safety (buffer overflow, use-after-free, integer overflow), and risky data exposure. The model learns, given a code snippet or a diff, to name the vulnerability (its class / CWE), point to the line, say why it is exploitable, and give the fix — the output shape the security-review and code-review skills already ask for.

Out of scope: writing exploits, offensive tooling, evasion. The data is (vulnerable code → defensive analysis); nothing teaches exploitation.

Data

OKLM-controlled, written for the stack:

  1. The corpus (tests/security/corpus): 81 snippets in C, Python and JavaScript. 40 have exactly one vulnerability, of 12 classes (CWE-22, 78, 79, 89, 95, 134, 190, 330, 416, 502, 787, 798; three or four snippets a class), and each has a fixed twin with the same code minus the flaw, so the model learns not to cry wolf; one more clean snippet has nothing to find. manifest.txt gives each snippet's split, CWE, line, class regex and the name it is shown under: a twin is shown under its vulnerable original's name, so within a pair only the code differs (a file named sql_injection.py or find_user_safe.py gives the answer away; the first captures showed it). No comment hints at the flaw. One vulnerable snippet a class and its twin are held out (24 snippets); the other 57 are for training.
  2. Reference answers (answers/FILE.md): for each vulnerable snippet the class and CWE, the line, why it is exploitable and the fix; for each clean one "No findings." and why. answers/FILE.think.md holds, for the snippets the model got wrong, the reasoning that reaches the reference answer, put in the final turn's think block.
  3. Capture (tools/make_security_data.sh, tools/security_data.py): the agent reviews each training snippet with oklm agent --save-training, in the exact in-process shape. A conversation whose answer is right (the class and the line, or "no findings" on a clean snippet) is kept as written; a wrong one keeps its turns (the snippet read, the code shown) and gets the reference answer as its last turn, after its reasoning when there is one; --repeat-ref N weights those conversations (the trainer samples conversations at random, so a copy is a weight). Keeping only the right ones (the MCP T1 rule) would keep only what the model already does: the MCP rounds showed it teaches nothing.

The capture writes build/security-data/{train,val}.jsonl. Content is the stack's own; no external corpus is imported (licence and quality stay controlled). Public vulnerability sets (Juliet, Big-Vul, DiverseVul) may be surveyed later for coverage of classes the seed corpus misses, mined for the shape of a snippet, never imported blindly.

Recipe

  • Base: the normal Qwen3-8B (Qwen3-8B-Q4_K_M.gguf or Q8_0), frozen and quantized (--quantized-base, item 18), so the run fits 20 GB.
  • Adapter: LoRA (R7), --lora 16 --lora-targets q,k,v,o,gate,up,down (the review task is reasoning-heavy, so the MLP is included), alpha = the rank. The adapter is a llama.cpp-format file, served with oklm serve --lora-module or merged with oklm lora-merge.
  • Format: the captured in-process shape (the security-review skill's system prompt, the code as a user turn, the finding as the assistant turn), as the MCP plan's decision 1.
  • Run: oklm train --lora 16 --quantized-base -m Qwen3-8B-Q4_K_M.gguf --chat-data build/security-data/train.jsonl --out build/security/adapter: --chat-data trains with the loss on the assistant turns only (training.md), so the run learns the review, not the vulnerable code it is shown.

Evaluation

  • Detection, held-out: snippets the training never saw, measured like tools/eval_review.sh — the fraction whose vulnerability the model names with the right class and line (target: beat the base model's rate; the base Qwen3-8B already finds 6/6 seeded C bugs at medium, so the eval uses a harder held-out set across classes).
  • False positives, held-out clean snippets: the fraction wrongly flagged (must not rise over the base).
  • No regression: the agent's non-security evals (tools/eval_agent.sh) unchanged, so the adapter does not dull general coding.

tools/eval_security.sh runs the detection and false-positive measures on the held-out snippets and writes build/eval-security/report.md; tools/security_train.sh measures the base, trains, measures the adapter and compares them. The judge: a vulnerable snippet is found when the answer names the class, whatever else it reports; the line is credited within one line; a clean snippet passes when the answer says there is nothing to find and cites no CWE. Measure greedily (-- --temp 0): the agent samples at 0.8 by default, and on 12 snippets one answer is 8 points, so a sampled run cannot tell an adapter from chance.

Results (Qwen3-8B Q4_K_M, OpenCL, 2026-11-07)

The base. Greedy, on the held-out snippets: 9 of 12 found (7 with the line), 6 of 12 clean snippets flagged. On the training snippets: 22 of 28 found, 13 of 29 clean ones flagged. Over-flagging is the weakness, not detection: fixed code is reported for vulnerabilities it no longer has (%63s called an overflow, textContent called XSS) or for CWEs that do not apply (NULL checks, HTTP rather than HTTPS, a missing salt). One finding was the corpus's: a "clean" twin hashed passwords with unsalted SHA-256 (both versions now use bcrypt).

Data. 57 conversations: 39 kept as written, 18 with the reference answer (15 of them for clean twins); 52 for training, 5 held out.

Training. LoRA rank 16, alpha 16, on q, k, v, o, gate, up and down of the frozen Q4_K_M base (--quantized-base --recompute), sequences of 6144 tokens, batch 1 with 2 accumulated, 50 steps at 1e-4 (cosine to 1e-5): 95 s a step, 80 minutes. The held-out loss on the assistant tokens fell from 0.309 to 0.166.

The adapter, greedy. Held-out: 9 of 12 found (7 with the line), 6 of 12 flagged: the same totals as the base, with 8 verdicts changed, 4 each way. Training snippets: 24 of 28 found, 11 of 29 flagged; on the 15 whose reference answer it trained on, 6 right, as the base. The run did not learn its own corrections: the loss fell, the behaviour did not change.

Why, and the next run. The corrections are 15 conversations among 52 (about 30 passes in all), and the 37 the model already gets right dominate the loss. And a reference answer follows an empty <think> block, while at run time the model reasons at length first and the false positive is argued there: the target trains the last sentence, not the reasoning that reaches it. The next run gives each reference answer a short reasoning that reaches the right verdict, repeats the corrections, and trains longer (or at a higher rate) until the training snippets themselves are right; held-out gains are measured only after that. More snippets come after.

Harness findings. The secret masker hid password= followed by any value, an expression too (password=os.environ[...] read as password=**********), so the model saw a hard-coded secret that was not there; the masker now masks literals only (2026-11-07). With the first adapter, one review thought past the 300 s limit.

Second run (2026-11-07)

Changes. Each of the 18 corrected conversations got a short reasoning reaching the reference verdict (answers/FILE.think.md) in place of the empty think block; the corrections repeated 3 times (82 training conversations, 5 held out); one twin fixed (thumbnail_spawn.c now refuses names starting with -, which convert would read as options); 100 steps at 2e-4 (cosine to 2e-5), the rest as the first run: 94 s a step, 2 h 37 min. The held-out loss on the assistant tokens went from 0.530 to 0.290 at step 60, then rose to 0.312 at step 100 while the training loss reached 0.03 to 0.10: the run overfits its 82 conversations past step 60.

The adapter, greedy (tools/security_train.sh, the base's greedy reviews from the first run reused):

Split Model Found (class) Found with the line Clean flagged
held-out base 9 of 12 7 of 12 6 of 12
held-out LoRA 6 of 12 6 of 12 2 of 12
training base 22 of 28 21 of 28 14 of 29
training LoRA 26 of 28 25 of 28 9 of 29

On the 18 corrected training snippets, 11 right against the base's 5: this run learned its corrections. On held-out code it moved the other way too: four clean twins no longer flagged (packet_checked.c, banner_fmt.c, frame_sized.c, settings_literal.py), three vulnerabilities newly missed (vec.c, otp.c, admin_auth.c), and a confident wrong "No findings" with plausible reasoning on frame.c (the 16-bit sum that wraps). Right answers on held-out: 16 of 24 against the base's 15, about even.

Why, and the next run. 15 of the 18 corrections were clean twins, so the run mostly taught "this code is fine"; it learned that, more widely than intended. Next: balanced corrections (a reasoning written for every vulnerable snippet the model misses, and for a share of those it finds, so the vulnerable side weighs as much as the clean side), the adapter taken at the lowest held-out loss (about step 60) rather than the last step, and more snippets per class (three or four now) before measuring held-out gains again: on 12 held-out pairs one answer is 8 points.

Risks

Risk Signal Mitigation
Dual-use drift the data drifts toward exploitation the keep rule accepts only defensive findings (class + line + fix); no exploit text is a label
Over-flagging false positives rise clean snippets in both the training set and the eval
Overfitting to the corpus high on seen classes, low on held-out held-out snippets of every class in the eval
Base identity / VRAM run fails to fit the quantized frozen base (item 18), as the MCP plan
Format ≠ inference low loss, poor review at run time train the captured in-process shape (MCP plan decision 1)