dd8c907800
macOS/Apple Silicon port: NEON kernels + platform shim (PR #1 by RDouglasSharp)
2026-07-06 22:01:12 +02:00
7a5ffd192d
async I/O: WILLNEED readahead of the next expert block while computing the current one
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 19:56:24 +02:00
757b0a8369
MLA weight absorption for decode (S<=4): no per-token k/v reconstruction — score via (W_K^T q)·latent, ctx via W_V(Σ a·latent). Validated exact: f32 TF 32/32, gen 20/20 with ABSORB=1 forced
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 19:54:51 +02:00
741f49a1ca
integer-dot kernels (Q8_0-style activations, AVX2 maddubs): int8 always, int4 when S>=2 — measured 1.4-2.5x
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 19:50:02 +02:00
2dd6800aea
sampling defaults for int4 reality: temp 0.7, nucleus 0.90 (official 1.0/0.95 tuned for full precision; the int4 tail is quantization noise)
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 19:30:34 +02:00
1a8d2bcff3
prefill progress: engine reports layer N/78 on stderr, coli spinner shows it live
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 18:18:50 +02:00
3278ea78d0
reserve 2.5 GB for the page cache in the RAM budget: starving it drops buffered reads 800->180 MB/s (measured)
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 18:00:35 +02:00
e982d2930e
autopin: pin share scales with usage-history confidence
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 17:26:08 +02:00
afcd71ab1c
surface MTP/USAGE status in coli chat banner (stderr)
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 17:03:49 +02:00
3e88e37ba2
learning cache + true sampling + DSA indexer extraction mode
...
- Learning cache: expert usage persists in <SNAP>/.coli_usage across sessions
(atomic save every turn); at startup the hottest experts are auto-pinned in
RAM with half the expert budget (AUTOPIN=0 disables). The engine gets faster
the more you use it.
- Sampling: temperature + nucleus (official 1.0/0.95 defaults in chat; TEMP=0
= greedy). MTP/n-gram speculation stays lossless via rejection sampling
(accept draft w.p. p(draft); on reject resample with draft banned).
- coli: --temp flag.
- Converter: --indexer mode extracts DSA lightning-indexer weights
(resumable; needed for future sparse attention beyond 2048 ctx).
- pin_load/stats include the MTP row; usage histogram covers layer 78.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 12:29:13 +02:00
257c4d0a8b
serve: :piu resumes a truncated answer from live KV; ngen default 1024 (stops end turns now)
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 11:54:47 +02:00
1ae22a6135
colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts
...
Engine (c/glm.c): MLA attention with compressed KV, sigmoid noaux_tc router,
int8/int4/int2 quant kernels (AVX2), per-layer LRU expert cache + pinned
hot-store, batch-union MoE, native MTP speculative decoding (lossless),
multi-stop + official chat template, RAM auto-budget from MemAvailable.
Tokenizer: byte-level BPE in C. Tooling: coli CLI, disk-safe FP8→int4
converter, tiny-random oracle validation (TF 32/32, greedy 20/20).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-05 20:57:25 +02:00
RDouglas