Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123)

Rebased onto current dev, split into 3 logical parts (all validated):
1. CPU portability (serve-mode _O_BINARY pipe fix — stock main hangs on MinGW without it; RAM detection cap 0->9/layer; POSIX guards for select/mmap/madvise; warmup script).
2. AVX-VNNI 128-bit int8/int4 dot kernel (Alder Lake+/Meteor Lake+), bit-identical to AVX2 (author-verified on Meteor Lake; compiles out to AVX2 elsewhere) + _mm256_extracti128_si256 typo fix that blocked -march=native.
3. CUDA DLL via LoadLibrary, gated behind CUDA_DLL=1 (host never links cudart; silent CPU fallback if absent; author-verified on RTX 5070 Ti).

Validated here: make check 59/59, oracle 32/32 TF, Windows cross-compile clean + glm.exe loads+runs via WSL interop. Fixes the #123 Windows build failure.
This commit is contained in:
woolcoxm
2026-07-13 14:54:30 -04:00
committed by GitHub
parent afc259c599
commit 2319b942d2
10 changed files with 644 additions and 30 deletions
+14 -1
View File
@@ -6,7 +6,14 @@ import tempfile
import unittest
from pathlib import Path
from resource_plan import GB, analyze_model, build_plan, environment_for_plan, format_plan
from resource_plan import (
GB,
analyze_model,
build_plan,
environment_for_plan,
format_plan,
memory_available,
)
def write_shard(path, tensors):
@@ -53,6 +60,12 @@ class ResourcePlanTest(unittest.TestCase):
self.assertEqual(info["expert_count"], 2)
self.assertEqual(info["per_cap_bytes"], 60)
def test_memory_available_is_positive(self):
# Regression: on native Windows CPython, /proc/meminfo does not exist,
# so the Linux-only path returned 0 and the expert cache was sized to
# 0 slots/layer. The value must be a sane positive number of bytes.
self.assertGreater(memory_available(), 0)
def test_builds_bounded_three_tier_plan(self):
gpus = [{"index": 0, "name": "test-gpu", "total_bytes": 12 * GB,
"free_bytes": 10 * GB}]