Windows native port: serve-mode pipe fix + RAM detection + POSIX guards, AVX-VNNI kernel, gated CUDA DLL (#131, fixes #123)
Rebased onto current dev, split into 3 logical parts (all validated): 1. CPU portability (serve-mode _O_BINARY pipe fix — stock main hangs on MinGW without it; RAM detection cap 0->9/layer; POSIX guards for select/mmap/madvise; warmup script). 2. AVX-VNNI 128-bit int8/int4 dot kernel (Alder Lake+/Meteor Lake+), bit-identical to AVX2 (author-verified on Meteor Lake; compiles out to AVX2 elsewhere) + _mm256_extracti128_si256 typo fix that blocked -march=native. 3. CUDA DLL via LoadLibrary, gated behind CUDA_DLL=1 (host never links cudart; silent CPU fallback if absent; author-verified on RTX 5070 Ti). Validated here: make check 59/59, oracle 32/32 TF, Windows cross-compile clean + glm.exe loads+runs via WSL interop. Fixes the #123 Windows build failure.
This commit is contained in:
@@ -6,7 +6,14 @@ import tempfile
|
||||
import unittest
|
||||
from pathlib import Path
|
||||
|
||||
from resource_plan import GB, analyze_model, build_plan, environment_for_plan, format_plan
|
||||
from resource_plan import (
|
||||
GB,
|
||||
analyze_model,
|
||||
build_plan,
|
||||
environment_for_plan,
|
||||
format_plan,
|
||||
memory_available,
|
||||
)
|
||||
|
||||
|
||||
def write_shard(path, tensors):
|
||||
@@ -53,6 +60,12 @@ class ResourcePlanTest(unittest.TestCase):
|
||||
self.assertEqual(info["expert_count"], 2)
|
||||
self.assertEqual(info["per_cap_bytes"], 60)
|
||||
|
||||
def test_memory_available_is_positive(self):
|
||||
# Regression: on native Windows CPython, /proc/meminfo does not exist,
|
||||
# so the Linux-only path returned 0 and the expert cache was sized to
|
||||
# 0 slots/layer. The value must be a sane positive number of bytes.
|
||||
self.assertGreater(memory_available(), 0)
|
||||
|
||||
def test_builds_bounded_three_tier_plan(self):
|
||||
gpus = [{"index": 0, "name": "test-gpu", "total_bytes": 12 * GB,
|
||||
"free_bytes": 10 * GB}]
|
||||
|
||||
Reference in New Issue
Block a user