woolcoxm 2319b942d2 Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123) ...

Rebased onto current dev, split into 3 logical parts (all validated):
1. CPU portability (serve-mode _O_BINARY pipe fix — stock main hangs on MinGW without it; RAM detection cap 0->9/layer; POSIX guards for select/mmap/madvise; warmup script).
2. AVX-VNNI 128-bit int8/int4 dot kernel (Alder Lake+/Meteor Lake+), bit-identical to AVX2 (author-verified on Meteor Lake; compiles out to AVX2 elsewhere) + _mm256_extracti128_si256 typo fix that blocked -march=native.
3. CUDA DLL via LoadLibrary, gated behind CUDA_DLL=1 (host never links cudart; silent CPU fallback if absent; author-verified on RTX 5070 Ti).

Validated here: make check 59/59, oracle 32/32 TF, Windows cross-compile clean + glm.exe loads+runs via WSL interop. Fixes the #123 Windows build failure.

2026-07-13 20:54:30 +02:00

..

audit_win_shims.c

Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40)

2026-07-11 12:59:49 +02:00

bench_tensor_core.cu

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_backend_cuda.cu

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_backend_metal.mm

Metal backend (Apple Silicon): batched experts + fused attention on GPU, unified-memory zero-copy, gated behind COLI_METAL — 2.06 tok/s M5 Max (#72, #87, #103)

2026-07-13 08:57:10 +02:00

test_benchmark_cuda_fixture.py

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

test_cli_output.py

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00

test_decode_batch

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_decode_batch.c

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_doctor.py

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_grammar.c

Grammar-forced speculative drafts: GBNF grammar as a third draft source, guaranteed-accepted forced spans, lossless + opt-in (#48, #70)

2026-07-12 01:35:39 +02:00

test_i4_acc512

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_i4_acc512.c

expert dot: AVX-512 int4→float accumulator, runtime-switchable (I4_ACC512), +7% CPU-heavy decode, more accurate than AVX2 order (#95)

2026-07-13 15:24:25 +02:00

test_idot

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_idot.c

VSX integer-dot kernels for POWER8 (12.8x int8 / 7.6x int4 over scalar, #ifdef-gated, x86 path untouched) (#98)

2026-07-12 22:44:56 +02:00

test_json.c

Local project checks and contribution templates: make check, dependency-free C/python tests, issue forms + blank issues, CONTRIBUTING (#20)

2026-07-10 08:56:41 +02:00

test_openai_server.py

openai server: type tool arguments from schema + implement tool_choice, 9 new tool-calling tests (#118)

2026-07-13 16:11:31 +02:00

test_resource_plan.py

Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123)

2026-07-13 20:54:30 +02:00

test_st.c

Local project checks and contribution templates: make check, dependency-free C/python tests, issue forms + blank issues, CONTRIBUTING (#20)

2026-07-10 08:56:41 +02:00

test_tier.c

Unify continuous batching + heterogeneous runtime: decode batching, physical-core planning, disjoint VRAM/RAM placement, topp-policy warning (CPU-validated, CUDA on 6x5090) (#68)

2026-07-13 14:30:36 +02:00

test_tok.c

Translate user-facing runtime output to English, machine prefixes preserved, + CLI output test (#67, #85)

2026-07-12 13:38:40 +02:00