Dennis Paul d439ac8680 attention: size per-thread score buffer by the batch's true max nt, not Tk+1 — fixes serve heap-overflow (#117) ...

step_decode_batch (run_serve_mux) passes per-slot positions[]/kv_start, so nt=pos+1-st0 can exceed the old Tk+1 cap -> heap-buffer-overflow on the first serve request (ASan-confirmed, dnnspaul). sc_cap now = max nt over the batch, counted exactly as the write loop (per-slot positions/kv_start + DSA top-k). Non-batched paths reduce to the correct cap (MTP kv_start=-1 -> Tk+1). Validated: make check 58/58, oracle 32/32 TF, build 0-warning; real-model serve completes with ASan clean.

2026-07-13 19:42:24 +02:00

..

scripts

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tests

openai server: type tool arguments from schema + implement tool_choice, 9 new tool-calling tests (#118)

2026-07-13 16:11:31 +02:00

tools

tools: quant_ablation.py — engine-free A/B of any quantization scheme vs fp16 (fake-quant, isolates weight error) (#108, #115)

2026-07-13 14:54:55 +02:00

.gitignore

Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40)

2026-07-11 12:59:49 +02:00

backend_cuda.cu

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

backend_cuda.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

backend_metal.h

Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103)

2026-07-13 08:57:10 +02:00

backend_metal.mm

Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103)

2026-07-13 08:57:10 +02:00

coli

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

compat.h

Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40)

2026-07-11 12:59:49 +02:00

decode_batch.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

doctor.py

coli doctor: read-only setup/health diagnostics (path, config, shards, disk, RAM budget, placement) (#33)

2026-07-11 19:39:13 +02:00

glm.c

attention: size per-thread score buffer by the batch's true max nt, not Tk+1 — fixes serve heap-overflow (#117)

2026-07-13 19:42:24 +02:00

grammar.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

iobench.c

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

json.h

colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts

2026-07-05 20:57:25 +02:00

Makefile

expert dot: AVX-512 int4→float accumulator, runtime-switchable (I4_ACC512), +7% CPU-heavy decode, more accurate than AVX2 order (#95)

2026-07-13 15:24:25 +02:00

olmoe.c

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

openai_server.py

openai server: type tool arguments from schema + implement tool_choice, 9 new tool-calling tests (#118)

2026-07-13 16:11:31 +02:00

ref_glm.json

colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts

2026-07-05 20:57:25 +02:00

resource_plan.py

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

setup.sh

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

st.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tier.h

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

tok_unicode.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

tok.h

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00