danielkrause/colibri-strix
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Commit Graph

  • 90fe263aab fix: thinking hart deaktiviert + leere think-tags entfernt (Crash-Fix) + Terminal-Chat-Client main cody 2026-07-15 14:52:44 +02:00
  • 6989590ab1 perf: znver5 + LTO + OMP 32T + GPU DENSE/ASYNC (Strix Halo optimiert) cody 2026-07-14 23:56:35 +02:00
  • 53c37ec918 Initial: Colibri upstream + HIP/ROCm port (Strix Halo / Radeon 8060S) cody 2026-07-14 23:53:20 +02:00
  • 6d3ed7e62b README: GB10 (DGX Spark) row (#136) — 0.50 tok/s warm, matmul-bound, unified memory; confirms #76 fix on aarch64 (TF 32/32 + greedy 20/20 on NEON and CUDA sm_121) JustVugg 2026-07-13 21:13:40 +02:00
  • 086b2dfb87 README: native-Windows warm-cache row (#128, 0.03->0.5 tok/s) — honest label per #131 (pipe+RAM fixes, no GPU) JustVugg 2026-07-13 21:00:50 +02:00
  • 2319b942d2 Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123) woolcoxm 2026-07-13 14:54:30 -04:00
  • afc259c599 Makefile: detect Windows from $(OS)=Windows_NT before uname so make works from native PowerShell/CMD (#129) bopof 2026-07-13 20:47:23 +02:00
  • 5dd7503ee7 docs: Metal M5 Max perf report — OMP hot-team spin (#77) throttles Apple GPU, NO_OMP+PIPE recovers to 2.24 tok/s (#116) RDouglas 2026-07-13 19:47:19 +01:00
  • f8c0552c6d quant_ablation: add rotation preconditioning (-rot, QuaRot/QuIP#) and int3 schemes (#132, #81) ZacharyZcR 2026-07-14 02:45:34 +08:00
  • 1f0e1b7076 olmoe: reject quant_bits outside 2..8 (fixes degenerate output, #134) + correct ref.json to OLMoE-0125-Instruct oracle (#133) JustVugg 2026-07-13 20:33:47 +02:00
  • 4418e16db4 README: benchmark rows for Gen5-NVMe 9950X3D2+5090 (1.23 tok/s, #120) and first Strix Halo datapoint (1.10 tok/s, #124) JustVugg 2026-07-13 20:26:52 +02:00
  • 5f49a5adca coli: honor --gpu/--vram without --auto-tier, fail-fast on CPU-only binary (#125, fixes #121) ZacharyZcR 2026-07-14 01:47:34 +08:00
  • d439ac8680 attention: size per-thread score buffer by the batch's true max nt, not Tk+1 — fixes serve heap-overflow (#117) Dennis Paul 2026-07-13 19:42:24 +02:00
  • 15bf8cac6b openai server: type tool arguments from schema + implement tool_choice, 9 new tool-calling tests (#118) Dennis Paul 2026-07-13 16:11:31 +02:00
  • 4b1d0e3a57 expert dot: AVX-512 int4→float accumulator, runtime-switchable (I4_ACC512), +7% CPU-heavy decode, more accurate than AVX2 order (#95) ZacharyZcR 2026-07-13 21:24:25 +08:00
  • 97c756a064 tools: quant_ablation.py — engine-free A/B of any quantization scheme vs fp16 (fake-quant, isolates weight error) (#108, #115) Dennis Paul 2026-07-13 14:54:55 +02:00
  • cbd599024e Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68) ZacharyZcR 2026-07-13 20:30:36 +08:00
  • 98759bfc40 Merge main into dev: native-Windows row (#113) JustVugg 2026-07-13 13:48:56 +02:00
  • cb53589c97 README: first native-Windows datapoint (#113, i5-12600K MinGW, MTP int8 57% — port validated) JustVugg 2026-07-13 13:48:48 +02:00
  • 05df180300 Merge main into dev: quality bench result + M4 Pro row + bench deps (#107,#108) JustVugg 2026-07-13 09:10:18 +02:00
  • 5254470f95 README: first quality benchmark result (#108, 62.5% with confound caveats + OLMoE A/B as decisive test), bench pip deps (tokenizers datasets), M4 Pro Metal row (#107) JustVugg 2026-07-13 09:09:16 +02:00
  • 3716e4006a Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103) Code Arranger 2026-07-12 23:57:10 -07:00
  • 342ceaf368 attention: heap per-thread score buffers — fix stack overflow past 8192 ctx on non-DSA snapshots (both absorb + dense paths, oracle-exact) (#110) John Balis 2026-07-12 23:52:58 -07:00
  • 4db6af60f8 Merge main into dev: EPYC 7443 datapoint (#104) JustVugg 2026-07-12 23:42:13 +02:00
  • a78a06fc5a README: EPYC 7443 430GB datapoint (#104) — hit 98%, disk eliminated, RAM-bandwidth+matmul bound; evidence lower-bit quant helps RAM-bound hosts too JustVugg 2026-07-12 23:42:05 +02:00
  • 46c54501e1 Merge main into dev: M5 Max Metal 2.06 record (#103) JustVugg 2026-07-12 23:28:36 +02:00
  • a9f3f35d21 README: M5 Max Metal 2.06 tok/s — new record, coherent output, on the pre-rebase branch (#103) JustVugg 2026-07-12 23:28:17 +02:00
  • 8ce825f6d4 Merge main into dev: generalized reproducibility note (#100) JustVugg 2026-07-12 23:13:15 +02:00
  • 92011def07 README: generalize the reproducibility note (#100) — quantized-kernel rounding forks greedy across MTP/CUDA/batched, not just MTP; exact-mode recipe DRAFT=0 IDOT=0 COLI_CUDA=0 JustVugg 2026-07-12 23:12:46 +02:00
  • f8a950f0a8 Merge main into dev: README int8-MTP mirror lead (#8,#102) JustVugg 2026-07-12 23:08:57 +02:00
  • 8e526c68a9 README: lead with the int8-MTP mirror, make the int4-head 0%-acceptance trap unmissable (#8, #102) — the #1 support confusion JustVugg 2026-07-12 23:08:36 +02:00
  • 0c327bbeb1 Merge main into dev: corrected 9800X3D benchmark (#101) JustVugg 2026-07-12 22:49:47 +02:00
  • 2baac89fee README: corrected 9800X3D benchmark row (#101) — 10.5 GB/s VHDX still disk-bound, CUDA tier ~0% when AVX-512 CPU matches the 5090; CUDA value is CPU-dependent JustVugg 2026-07-12 22:49:29 +02:00
  • bc6bc9c250 VSX integer-dot kernels for POWER8 (12.8x int8 / 7.6x int4 over scalar, #ifdef-gated, x86 path untouched) (#98) AutoJanitor 2026-07-12 15:44:56 -05:00
  • 9bba681ae2 perf(pilot): PILOT_REAL real cross-layer expert loads, independent from PIPE pool (data-driven: two pools beat layered/unified on both disk- and matmul-bound hosts) (#78) Rodolfo Hansen 2026-07-12 22:39:20 +02:00
  • 69e3a3a61b Merge main into dev: README honesty fixes (#100,#101) + #96 tool calling + #97 ppc64le (landed on main by mistake, syncing dev) JustVugg 2026-07-12 22:25:28 +02:00
  • 0f9f99cc29 OpenAI tool calling for GLM-5.2: gated tool declaration + parse_tool_calls, non-tool path preserved, opt-in salvage (#96) RDouglas 2026-07-12 21:14:41 +01:00
  • 6aaa8fc37a Makefile: Linux PowerPC (ppc64le) build support — -mcpu, scalar fallback, x86 path untouched (#97) AutoJanitor 2026-07-12 15:13:32 -05:00
  • f1fbbca352 README honesty fixes: MTP not byte-identical to greedy (shape-dependent kernels, #100); VHDX ceiling was this drive not the virtualization layer (#101, community measured 10.5 GB/s) JustVugg 2026-07-12 22:08:13 +02:00
  • 3fb5a00106 docs: GLM-5.2 6x RTX 5090 full-residency experiment — 6.84 tok/s, honest dead-ends log (#94) ZacharyZcR 2026-07-13 04:06:51 +08:00
  • f38e74efe7 docs: adopt dev integration branch — PRs target dev, batched into stable main (ZacharyZcR's suggestion) JustVugg 2026-07-12 14:30:03 +02:00
  • e945b43842 README: Metal backend M5 Max benchmark — 1.83 tok/s, fastest datapoint yet (#72, #87) JustVugg 2026-07-12 14:08:40 +02:00
  • 928fd67cc2 README: correct M5 Max disk number (14.2 GB/s was cache-influenced ~4 GB/s real) + iobench cold-read caveat, macOS F_NOCACHE limitation (#86) JustVugg 2026-07-12 14:04:11 +02:00
  • 03d9a23fe4 perf(moe): overlap NVMe expert loads with matmul via opt-in PIPE I/O pool (default off, oracle-exact) (#79) Rodolfo Hansen 2026-07-12 13:44:34 +02:00
  • 704ed49c16 perf(omp): keep OpenMP worker team hot across tiny per-expert matmul regions (seed + re-exec once, fully overridable) (#77) Rodolfo Hansen 2026-07-12 13:40:29 +02:00
  • 2416bc9079 Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85) Sidd 2026-07-12 17:08:40 +05:30
  • 6e7aa6f92e Guard against running the tiny oracle against a real model (#76) JustVugg 2026-07-12 13:35:00 +02:00
  • 5198feec7c Tauri desktop shell (reuses web/ frontend, connects to OpenAI-compatible endpoint, isolated in desktop/) (#24) ZacharyZcR 2026-07-12 07:40:21 +08:00
  • 57730f6196 Web UI: runtime console + KV session panel, with runtime/storage libs and tests (#32) ZacharyZcR 2026-07-12 07:40:01 +08:00
  • d7ffdc45be Add flake.nix for reproducible NixOS builds (engine + converter python env) (#66) Maikel Frias Mosquea 2026-07-12 01:35:56 +02:00
  • cec7d6b648 Grammar-forced speculative drafts: GBNF grammar as a third draft source, guaranteed-accepted forced spans, lossless + opt-in (#48, #70) Fabio Rovai 2026-07-12 00:35:39 +01:00
  • 1bab3243f7 iobench: Windows correctness — aligned free, 64-bit totals, full-range random offsets (also improves Linux offset coverage) (#64) nalepy 2026-07-11 20:19:40 -03:00
  • 8f5f3e3a2b coli doctor: read-only setup/health diagnostics (path, config, shards, disk, RAM budget, placement) (#33) ZacharyZcR 2026-07-12 01:39:13 +08:00
  • 1bdaeee82e setup.sh: detect libomp by dylib presence, not brew prefix (false-positive fix by fabio-rovai, #47) JustVugg 2026-07-11 13:02:34 +02:00
  • 89d95fc73b Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40) nalepy 2026-07-11 07:59:49 -03:00
  • 7d8d5f3109 Reduce allocation overhead in quantized matmul and MoE routing (thread-local scratch, trimmed temporaries) (#43) 0xhis 2026-07-11 03:59:01 -07:00
  • a5fc89e88f README: Strix Halo warm numbers (#39) — 0.16 -> 0.40 tok/s in five runs, learned pin 47.6 GB, fastest non-Apple datapoint JustVugg 2026-07-11 01:25:59 +02:00
  • ae8975a009 README: Strix Halo + Optane 905p datapoint (#39) — first Ryzen AI Max+, Optane latency invisible to 19MB-chunk reads JustVugg 2026-07-11 00:13:56 +02:00
  • c809bc8260 README: 9950X disk-swap datapoint (#31) — same machine, P3 vs 9100 PRO: x5.8 disk = x2.9 tokens, profile flips 66% disk -> 57% matmul JustVugg 2026-07-11 00:01:56 +02:00
  • 9e6e1ac327 Isolated sequence KV contexts: KVState extraction, KV_SLOTS serve contexts with per-slot persistence, budget-aware pool accounting (#29) ZacharyZcR 2026-07-10 22:19:43 +08:00
  • 76e858a80f Bounded FIFO inference scheduling: admission queue, OpenAI-shaped 429/503, queued-disconnect detection (#28) ZacharyZcR 2026-07-10 18:52:07 +08:00
  • 3cbc52ab00 coli plan: read-only Disk/RAM/VRAM placement planner (mirrors engine budget math, versioned --json, --auto-tier) (#27) ZacharyZcR 2026-07-10 18:51:23 +08:00
  • 1453dab7ae REPIN follows live session heat (decaying heat map; .coli_usage stays the persistent signal) + disk→RAM→VRAM promotion (#26) ZacharyZcR 2026-07-10 18:48:53 +08:00
  • 3e4d08b6bf OpenAI-compatible HTTP API: stdlib-only gateway over SERVE with KV prefix reuse across stateless requests (#21) ZacharyZcR 2026-07-10 17:04:56 +08:00
  • ea5f6fb914 Harden against hostile model files: config dim validation at parse + overflow-safe falloc (PR #25 report, done right) JustVugg 2026-07-10 10:53:04 +02:00
  • 1a909f3833 README: catch up with the last two days — DSA is done (was 'in progress'), KV persistence, PILOT, web/ UI section + repo layout, honest line count JustVugg 2026-07-10 10:12:40 +02:00
  • 13e8f09ffc Organize project tools and local workflows: c/tools, c/scripts, c/tests, root Makefile (flat C core untouched) (#22) ZacharyZcR 2026-07-10 16:08:39 +08:00
  • a2942b2172 Web interface (React/TS, shadcn, pure OpenAI-API client) under web/ (#23) ZacharyZcR 2026-07-10 16:07:29 +08:00
  • 8f99e12b5e KV-to-disk: conversations reopen WARM across engine restarts (.coli_kv, KVSAVE=0 opts out) JustVugg 2026-07-10 10:00:19 +02:00
  • 8a2e4439ba Local project checks and contribution templates: make check, dependency-free C/python tests, issue forms + blank issues, CONTRIBUTING (#20) ZacharyZcR 2026-07-10 14:56:41 +08:00
  • 99111993a4 README: Framework 13 warm numbers (0.37 tok/s, hit 66%, MTP 52% — #12), int8-MTP mirror clone link (#2), Epyc row column fix JustVugg 2026-07-10 07:46:18 +02:00
  • 077894210e LOOKA: routing predictability counters; PILOT: router-lookahead disk prefetch (async I/O thread); cap auto-raise to RAM budget (#12) JustVugg 2026-07-10 07:38:47 +02:00
  • 57706a0200 Tiered CUDA acceleration for routed experts (opt-in, CPU default untouched) + REPLAY fixture harness (#16) ZacharyZcR 2026-07-10 13:41:09 +08:00
  • 4ea9ddc0f0 README: community benchmark — Epyc 9654 ES, PCIe3 NVMe (0.31 tok/s, hit 35%) (#17) adapt-L 2026-07-10 01:41:06 -04:00
  • ed3916b0bb README: accurate SSD note (reads don't wear the drive; swap + thermals are the real concerns) JustVugg 2026-07-10 01:39:50 +02:00
  • c035f41778 MTP head is int8 by default (int4 head: 0-4% acceptance, unusable — measured in #8); README: honest MTP numbers + M5 Max community datapoint (#4, #5) JustVugg 2026-07-09 20:34:08 +02:00
  • ab6950876b RFC: live re-pin of the hot-store between turns (REPIN=n, opt-in, default off) (#11) RDouglas 2026-07-09 11:49:40 +01:00
  • 3daceaf5ec IDOT: NEON single-token int4 gate (g_i4s/I4S) — SDOT makes S=1 decode pay on Apple Silicon (+14%, expert-matmul -16%) (#10) RDouglas 2026-07-09 11:49:06 +01:00
  • 8040dac645 IDOT: AVX-512 VNNI kernel paths + batch-size gate for single-row int4 (+20% decode on Xeon) (#9) Dog279 2026-07-09 03:44:40 -07:00
  • cfd16b4c7e PIN: mlock pinned experts into physical RAM (macOS: stop the compressor evicting the hot-store) (#7) RDouglas 2026-07-09 11:44:11 +01:00
  • e74eb07e5a converter: multi-stream, Range-resume shard downloader (2-4x faster, no lost bytes) (#6) RDouglas 2026-07-09 11:43:00 +01:00
  • 996ae0e3cd README: add measured community benchmark (WSL2, 24 GB RAM — issue #2) (#3) DatPat 2026-07-08 10:20:37 +02:00
  • 4319dfdf91 Add model download huggingface section to README Vincenzo 2026-07-08 08:42:44 +02:00
  • 2bd886e889 coli chat: streaming markdown renderer — clean chat, no raw markers JustVugg 2026-07-07 13:15:48 +02:00
  • 492c3b6f63 Revert "coli: multi-model registry + picker (coli models / chat menu); README: MoE-only model roadmap (gpt-oss-120b next)" JustVugg 2026-07-07 13:03:07 +02:00
  • 289befb0fd coli: multi-model registry + picker (coli models / chat menu); README: MoE-only model roadmap (gpt-oss-120b next) JustVugg 2026-07-07 12:59:56 +02:00
  • b71d3884f6 coli bench: self-contained (venv python + auto-fetch datasets); README calls for a quality run on faster hardware JustVugg 2026-07-07 07:29:52 +02:00
  • 193d2ce92d DSA lightning indexer: full implementation with auto-detection JustVugg 2026-07-06 22:07:50 +02:00
  • dd8c907800 macOS/Apple Silicon port: NEON kernels + platform shim (PR #1 by RDouglasSharp) RDouglas 2026-07-06 21:01:12 +01:00
  • 8d7fbef39c README: today's engine work (integer kernels, MLA absorption, readahead, sampling, DSA status) JustVugg 2026-07-06 20:24:18 +02:00
  • 7a5ffd192d async I/O: WILLNEED readahead of the next expert block while computing the current one JustVugg 2026-07-06 19:56:24 +02:00
  • 757b0a8369 MLA weight absorption for decode (S<=4): no per-token k/v reconstruction — score via (W_K^T q)·latent, ctx via W_V(Σ a·latent). Validated exact: f32 TF 32/32, gen 20/20 with ABSORB=1 forced JustVugg 2026-07-06 19:54:51 +02:00
  • 741f49a1ca integer-dot kernels (Q8_0-style activations, AVX2 maddubs): int8 always, int4 when S>=2 — measured 1.4-2.5x JustVugg 2026-07-06 19:50:02 +02:00
  • 34fb900762 README: honest matmul-bound predictions (GFLOP math), updated sampling defaults JustVugg 2026-07-06 19:40:27 +02:00
  • 2dd6800aea sampling defaults for int4 reality: temp 0.7, nucleus 0.90 (official 1.0/0.95 tuned for full precision; the int4 tail is quantization noise) JustVugg 2026-07-06 19:30:34 +02:00
  • 1a8d2bcff3 prefill progress: engine reports layer N/78 on stderr, coli spinner shows it live JustVugg 2026-07-06 18:18:50 +02:00
  • 3278ea78d0 reserve 2.5 GB for the page cache in the RAM budget: starving it drops buffered reads 800->180 MB/s (measured) JustVugg 2026-07-06 18:00:35 +02:00
  • e982d2930e autopin: pin share scales with usage-history confidence JustVugg 2026-07-06 17:26:08 +02:00
  • b56d58d388 coli: user box adapts to multi-line messages; status lines one per row without paths JustVugg 2026-07-06 17:24:45 +02:00
  • afcd71ab1c surface MTP/USAGE status in coli chat banner (stderr) JustVugg 2026-07-06 17:03:49 +02:00