57706a0200
Tiered CUDA acceleration for routed experts (opt-in, CPU default untouched) + REPLAY fixture harness ( #16 )
...
* feat: add experimental CUDA backend for resident tensors
* feat: promote pinned experts to a bounded VRAM tier
* feat: preload the GPU expert tier at startup
* fix: harden CUDA backend failure handling
* feat: add deterministic multi-GPU tensor placement
* test: add deterministic CUDA benchmark fixture
* perf: make routed experts the default CUDA path
2026-07-10 07:41:09 +02:00
4ea9ddc0f0
README: community benchmark — Epyc 9654 ES, PCIe3 NVMe (0.31 tok/s, hit 35%) ( #17 )
...
I think I am just disk bandwidth limited, CPU's are at like 65% average. Might consider upgrading to 4x PCIe5 x4 ssds...
2026-07-10 07:41:06 +02:00
ed3916b0bb
README: accurate SSD note (reads don't wear the drive; swap + thermals are the real concerns)
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-10 01:39:50 +02:00
c035f41778
MTP head is int8 by default (int4 head: 0-4% acceptance, unusable — measured in #8 ); README: honest MTP numbers + M5 Max community datapoint ( #4 , #5 )
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-09 20:34:08 +02:00
996ae0e3cd
README: add measured community benchmark (WSL2, 24 GB RAM — issue #2 ) ( #3 )
...
First real datapoint for the 'Got a better machine?' section: disk iobench
plus stock and --topp 0.7 inference numbers, with the RAM-bound takeaway.
Co-authored-by: Claude Fable 5 <noreply@anthropic.com >
2026-07-08 10:20:37 +02:00
4319dfdf91
Add model download huggingface section to README
...
Added download instructions for the GLM-5.2 int4 model on Hugging Face.
2026-07-08 08:42:44 +02:00
492c3b6f63
Revert "coli: multi-model registry + picker (coli models / chat menu); README: MoE-only model roadmap (gpt-oss-120b next)"
...
This reverts commit 289befb0fd
2026-07-07 13:03:07 +02:00
289befb0fd
coli: multi-model registry + picker (coli models / chat menu); README: MoE-only model roadmap (gpt-oss-120b next)
...
Each architecture maps to its own engine binary (glm today; gptoss, qwenmoe
reserved). Registry in c/models.json (local, gitignored); chat shows a picker
when more than one model is installed. Dense models stay llama.cpp territory
- documented honestly in the README.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-07 12:59:56 +02:00
b71d3884f6
coli bench: self-contained (venv python + auto-fetch datasets); README calls for a quality run on faster hardware
...
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com >
2026-07-07 07:29:52 +02:00
8d7fbef39c
README: today's engine work (integer kernels, MLA absorption, readahead, sampling, DSA status)
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 20:24:18 +02:00
34fb900762
README: honest matmul-bound predictions (GFLOP math), updated sampling defaults
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 19:40:27 +02:00
b2855c0da1
coli convert: one command does model + MTP head; README documents the full zero-to-chat path
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-06 16:59:27 +02:00
8e8d5fb02b
Update README.md
2026-07-05 23:22:23 +02:00
ea27cd77fb
Update README.md
2026-07-05 23:18:00 +02:00
1e33e9b17d
Update README.md
2026-07-05 23:16:58 +02:00
31d91b2c5b
README: TUI pixel-art hummingbird as SVG logo banner
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-05 21:15:09 +02:00
20008742bf
README: try-it-on-better-hardware guide + predictions + support section; drop stage-0 research scripts from repo
...
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-05 21:02:46 +02:00
1ae22a6135
colibrì: pure-C GLM-5.2 (744B MoE) engine with disk-streamed experts
...
Engine (c/glm.c): MLA attention with compressed KV, sigmoid noaux_tc router,
int8/int4/int2 quant kernels (AVX2), per-layer LRU expert cache + pinned
hot-store, batch-union MoE, native MTP speculative decoding (lossless),
multi-stop + official chat template, RAM auto-budget from MemAvailable.
Tokenizer: byte-level BPE in C. Tooling: coli CLI, disk-safe FP8→int4
converter, tiny-random oracle validation (TF 32/32, greedy 20/20).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com >
2026-07-05 20:57:25 +02:00
a6deef8e44
Initial commit
2026-07-01 14:27:49 +02:00
ZacharyZcR