Windows 11 native port, phase 1: MinGW-w64 static build, compat shims, setup + docs (#40)

* fix: Windows port audit fixes — _FILE_OFFSET_BITS guard, O_BINARY st.h, getrusage peak, oracle diagnostic, setup.sh wmic Audit remediation (all MEDIUM issues fixed): - compat.h: compile-time #error if _FILE_OFFSET_BITS < 64 on _WIN32 - compat.h: COMPAT_O_RDONLY macro (O_RDONLY|O_BINARY on Windows, belt-and-braces) - st.h: use COMPAT_O_RDONLY in both open() call sites (plan §1 row 1) - compat.h: getrusage shim now uses PeakWorkingSetSize (ru_maxrss = peak, not current) - glm.c: oracle mismatch diagnostic — prints position/expected/got on TF failures - setup.sh: replace deprecated wmic with /proc/meminfo (MSYS2 provides it) - .gitignore: *.exe, glm_tiny/, olmoe_hf/, olmoe_i4/ LOW issues addressed: - _FILE_OFFSET_BITS guard prevents silent 32-bit off_t wrap at >4GB offsets - coli Windows venv path (Scripts/python.exe) fixed earlier - posix_fadvise do{}while(0) kept intentionally — no caller checks return code Verification: oracle 32/32, 27/27 Python tests, rename EEXIST, >4GB pread offset. * docs: add Windows 11 native port section to README - Toolchain: MinGW-w64 (winlibs or MSYS2), GCC 16.1 tested - Build instructions: make glm.exe, tiny oracle verification - Runtime: SNAP=..., coli chat, coli serve all work - Status: Phase 1 complete (compiles, correct, static-linked) - Update platform requirements to include Windows 11 natively
2026-07-11 07:59:49 -03:00
parent 7d8d5f3109
commit 89d95fc73b
13 changed files with 493 additions and 35 deletions
@@ -112,6 +112,44 @@ variables keep precedence over automatic values.

 The engine at runtime is pure C — python is only used by the one-time converter.

+### Windows 11 (native, no WSL)
+
+colibrì builds and runs natively on Windows 11 x86-64 with MinGW-w64. The port adds
+a `_WIN32` compatibility layer in `c/compat.h` that maps POSIX I/O to the Windows API
+(pread → ReadFile+OVERLAPPED, posix_fadvise no-op, aligned allocation, MoveFileEx rename,
+GlobalMemoryStatusEx RAM detection). All platform differences stay in `compat.h`; the
+engine source is unchanged.
+
+**Toolchain:** GCC via [winlibs](https://winlibs.com/) or MSYS2 MinGW-w64. Tested with
+GCC 16.1.0 (x86_64-ucrt-posix-seh).
+
+```powershell
+# One-time toolchain install (pick one):
+scoop install mingw-winlibs                    # portable, no shell needed
+# or: pacman -S mingw-w64-x86_64-gcc make     # via MSYS2
+
+# Build (from c/ directory):
+make glm.exe            # GLM-5.2 engine (static, no DLL dependencies)
+make olmoe.exe          # OLMoE engine (same shims)
+make iobench.exe        # disk I/O benchmark
+make test-c             # run C tests
+make test-python        # run Python tests (requires python)
+
+# Verify (tiny model, 2.4 MB):
+pip install torch transformers safetensors huggingface_hub
+python tools/make_glm_oracle.py                # generate tiny oracle
+SNAP=./glm_tiny TF=1 ./glm.exe 64 16 16        # expect "32/32 posizioni"
+
+# Run with real model:
+SNAP=D:\glm52_i4 ./glm.exe 64 4 16            # batch inference
+python coli chat --model D:\glm52_i4            # interactive chat
+python coli serve --model D:\glm52_i4            # OpenAI-compatible API
+```
+
+**Status:** Phase 1 complete (compiles, correct, static-linked). O_DIRECT (Phase 2),
+GPU via `LoadLibrary` on `coli_cuda.dll` (Phases G0–G2), and full-model validation
+are separate workstreams. See `PORT_WINDOWS_PLAN.md` for the full plan.
+
 ### OpenAI-compatible API

 `coli serve` keeps one model process loaded and exposes a text-only OpenAI-compatible
@@ -277,7 +315,7 @@ thrashing. Persistent `.coli_usage` remains the long-term signal and is not deca

 ## Got a better machine? Try it — here's what to expect

-colibrì was built on deliberately humble hardware (12 cores, 25 GB RAM, NVMe behind a WSL2 VHDX that caps random reads at ~1 GB/s). **Every one of those constraints is a knob your machine can turn up.** The engine needs: Linux (or WSL2), gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe (ext4 — never a network/9p mount).
+colibrì was built on deliberately humble hardware (12 cores, 25 GB RAM, NVMe behind a WSL2 VHDX that caps random reads at ~1 GB/s). **Every one of those constraints is a knob your machine can turn up.** The engine needs: Linux (or WSL2), macOS, or **Windows 11 natively (MinGW-w64)**; gcc with OpenMP, AVX2, ≥16 GB RAM, and the ~370 GB int4 model on a local NVMe (ext4/NTFS — never a network/9p mount).

 **How to test it, in order:**

@@ -0,0 +1,4 @@
+*.exe
+glm_tiny/
+olmoe_hf/
+olmoe_i4/
@@ -1,4 +1,7 @@
 UNAME_S := $(shell uname -s)
+MINGW    := $(findstring MINGW,$(UNAME_S))
+MSYS     := $(findstring MSYS,$(UNAME_S))
+IS_WIN   := $(MINGW)$(MSYS)

 ifeq ($(UNAME_S),Darwin)
 # --- macOS / Apple Silicon ---
@@ -17,6 +20,18 @@ OMPL    =
 endif
 CFLAGS  = -O3 $(OMPC) -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function
 LDFLAGS = -lm $(OMPL)
+EXE     =
+else ifneq ($(IS_WIN),)
+# --- Windows 11 x86-64 (MinGW-w64 / MSYS2) ---
+# GCC + libgomp + winpthreads: pthread, OpenMP, clock_gettime, opendir/readdir,
+# AVX2 intrinsics — tutto gratis, nessun porting.
+# ARCH default = x86-64-v3 (binario portabile con AVX2). Per max velocita'
+# su QUESTA macchina: make ARCH=native
+CC      = gcc
+ARCH   ?= x86-64-v3
+CFLAGS  = -D_FILE_OFFSET_BITS=64 -O3 -march=$(ARCH) -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function
+LDFLAGS = -lm -fopenmp -static
+EXE     = .exe
 else
 # --- Linux x86-64 (percorso originale, invariato) ---
 CC      = gcc
@@ -26,6 +41,7 @@ CC      = gcc
 ARCH   ?= native
 CFLAGS  = -O3 -march=$(ARCH) -fopenmp -Wall -Wextra -Wno-unused-parameter -Wno-misleading-indentation -Wno-unused-function
 LDFLAGS = -lm -fopenmp
+EXE     =
 endif

 # CUDA=1 adds an opt-in backend for resident tensors. The default build remains
@@ -37,20 +53,27 @@ CUDA_ARCH ?= native
 NVCCFLAGS ?= -O3 -std=c++17 -arch=$(CUDA_ARCH) -Xcompiler=-Wall,-Wextra
 PYTHON    ?= python3
 CUDA_OBJ  =
-TEST_BINS = tests/test_json tests/test_st tests/test_tier
+TEST_BINS = tests/test_json$(EXE) tests/test_st$(EXE) tests/test_tier$(EXE)
 ifeq ($(CUDA),1)
 ifeq ($(UNAME_S),Darwin)
 $(error CUDA=1 is supported only on Linux)
 endif
+ifneq ($(IS_WIN),)
+# GPU: stub only in Phase 1 (G0). G1 builds coli_cuda.dll with MSVC+nvcc.
+$(error CUDA=1 on Windows requires G1: build coli_cuda.dll with MSVC+nvcc (see PORT_WINDOWS_PLAN.md §8))
+endif
 CFLAGS   += -DCOLI_CUDA
 LDFLAGS  += -L$(CUDA_HOME)/lib64 -Wl,-rpath,$(CUDA_HOME)/lib64 -lcudart -lstdc++
 CUDA_OBJ  = backend_cuda.o
 endif

-all: glm
+all: glm$(EXE)

-glm: glm.c st.h json.h tok.h tok_unicode.h compat.h $(CUDA_OBJ)
-	$(CC) $(CFLAGS) glm.c $(CUDA_OBJ) -o glm $(LDFLAGS)
+# phony 'glm' → 'glm.exe' on Windows (so 'make glm' and 'coli build' work on every platform)
+glm: glm$(EXE)
+
+glm$(EXE): glm.c st.h json.h tok.h tok_unicode.h compat.h $(CUDA_OBJ)
+	$(CC) $(CFLAGS) glm.c $(CUDA_OBJ) -o glm$(EXE) $(LDFLAGS)

 backend_cuda.o: backend_cuda.cu backend_cuda.h
 	@command -v $(NVCC) >/dev/null 2>&1 || { echo "nvcc not found: set CUDA_HOME or NVCC" >&2; exit 1; }
@@ -58,23 +81,26 @@ backend_cuda.o: backend_cuda.cu backend_cuda.h

 cuda-test: backend_cuda.cu backend_cuda.h tests/test_backend_cuda.cu
 	@command -v $(NVCC) >/dev/null 2>&1 || { echo "nvcc not found: set CUDA_HOME or NVCC" >&2; exit 1; }
-	$(NVCC) $(NVCCFLAGS) backend_cuda.cu tests/test_backend_cuda.cu -o backend_cuda_test
-	./backend_cuda_test
+	$(NVCC) $(NVCCFLAGS) backend_cuda.cu tests/test_backend_cuda.cu -o backend_cuda_test$(EXE)
+	./backend_cuda_test$(EXE)

-olmoe: olmoe.c st.h json.h
-	$(CC) $(CFLAGS) olmoe.c -o olmoe $(LDFLAGS)
+olmoe$(EXE): olmoe.c st.h json.h compat.h
+	$(CC) $(CFLAGS) olmoe.c -o olmoe$(EXE) $(LDFLAGS)

 # binario portabile da distribuire su altre macchine x86-64
 portable:
-	$(MAKE) glm ARCH=x86-64-v3
+	$(MAKE) glm$(EXE) ARCH=x86-64-v3

-tests/test_json: tests/test_json.c json.h
+iobench$(EXE): iobench.c compat.h
+	$(CC) $(CFLAGS) iobench.c -o iobench$(EXE) $(LDFLAGS)
+
+tests/test_json$(EXE): tests/test_json.c json.h
 	$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)

-tests/test_st: tests/test_st.c st.h json.h compat.h
+tests/test_st$(EXE): tests/test_st.c st.h json.h compat.h
 	$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)

-tests/test_tier: tests/test_tier.c tier.h
+tests/test_tier$(EXE): tests/test_tier.c tier.h
 	$(CC) $(CFLAGS) $< -o $@ $(LDFLAGS)

 test-c: $(TEST_BINS)
@@ -92,7 +118,7 @@ check:
 	$(MAKE) test

 clean:
-	rm -f olmoe glm backend_cuda.o backend_cuda_test $(TEST_BINS)
+	rm -f olmoe$(EXE) glm$(EXE) iobench$(EXE) backend_cuda.o backend_cuda_test$(EXE) $(TEST_BINS)
 	rm -rf tests/__pycache__

-.PHONY: all cuda-test portable test-c test-python test check clean
+.PHONY: all glm cuda-test portable test-c test-python test check clean
@@ -21,9 +21,15 @@ Config via env o flag (validi anche dopo il sottocomando):
 """
 import os, sys, subprocess, argparse, json, time, signal, shutil, threading, re, codecs, tempfile, textwrap

+# Windows: forza output UTF-8 (console cp1252 tronca Unicode box-drawing/emoji)
+if sys.platform == "win32":
+    for s in (sys.stdout, sys.stderr):
+        try: s.reconfigure(encoding="utf-8")
+        except (AttributeError, OSError): pass
+
 HERE = os.path.dirname(os.path.abspath(__file__))
 TOOLS = os.path.join(HERE, "tools")
-GLM  = os.path.join(HERE, "glm")
+GLM  = os.path.join(HERE, "glm" + (".exe" if sys.platform == "win32" else ""))
 DEF_MODEL = os.environ.get("COLI_MODEL", "/home/vincenzo/glm52_i4")
 END   = b"\x01\x01END\x01\x01\n"
 READY = b"\x01\x01READY\x01\x01\n"
@@ -299,8 +305,11 @@ def cmd_info(a):
        av=int(re.search(r'MemAvailable:\s+(\d+)',mi).group(1))/1e6
        row("RAM", f"{tot:.0f} GB totali · {av:.1f} GB disponibili")
    except Exception: pass
-    fs=os.statvfs(a.model if os.path.isdir(a.model) else HERE)
-    row("disco", f"{fs.f_bavail*fs.f_frsize/1e9:.0f} GB liberi")
+    try:
+        fs = shutil.disk_usage(a.model if os.path.isdir(a.model) else HERE)
+        row("disco", f"{fs.free/1e9:.0f} GB liberi")
+    except OSError:
+        row("disco", "? GB (non disponibile)")
    row("motore", "pronto ✓" if os.path.exists(GLM) else "da compilare (coli build)")
    knobs=[]
    if a.ram: knobs.append(f"ram {a.ram}GB")
@@ -447,7 +456,7 @@ def cmd_bench(a):
    need_model(a.model)
    banner("bench")
    # python con `tokenizers`: l'ambiente del progetto se c'e', altrimenti quello corrente
-    venv_py=os.path.join(HERE,"mio_env","bin","python3")
+    venv_py = os.path.join(HERE, "mio_env", "Scripts" if sys.platform == "win32" else "bin", "python3")
    py = venv_py if os.path.exists(venv_py) else sys.executable
    tasks = ",".join(a.tasks) if a.tasks else "hellaswag,arc_challenge,mmlu"
    # dataset mancanti -> li scarica una volta (fetch_benchmarks.py li mette in --data come JSONL)
@@ -466,7 +475,7 @@ def cmd_bench(a):
 def cmd_convert(a):
    banner("convert")
    # python con torch/safetensors: l'ambiente del progetto se c'e', altrimenti quello corrente
-    venv_py=os.path.join(HERE,"mio_env","bin","python3")
+    venv_py = os.path.join(HERE, "mio_env", "Scripts" if sys.platform == "win32" else "bin", "python3")
    py = venv_py if os.path.exists(venv_py) else sys.executable
    base=[py, os.path.join(TOOLS,"convert_fp8_to_int4.py"),
          "--repo", a.repo, "--outdir", a.model, "--ebits", str(a.ebits), "--io-bits", str(a.io_bits)]
@@ -1,4 +1,5 @@
-/* compat.h — shim di portabilita' per piattaforme non-Linux (oggi: macOS / Apple Silicon).
+/* compat.h — shim di portabilita' per piattaforme non-Linux (oggi: macOS / Apple Silicon,
+ * Windows 11 x86-64 via MinGW-w64).
 * Su Linux questo header e' un NO-OP totale: nessun simbolo definito o ridefinito,
 * zero impatto sul percorso x86 esistente.
 * Regola: ogni differenza di piattaforma vive QUI; i .c restano puliti. */
@@ -47,4 +48,191 @@ static inline int compat_open_direct(const char *path){
 }
 #endif /* __APPLE__ */

+/* ===================================================================
+ * Windows 11 x86-64 (MinGW-w64 / MSYS2)
+ * ===================================================================
+ * pread         -> compat_pread  (ReadFile + OVERLAPPED su raw handle:
+ *                                  thread-safe, 64-bit offset, no CRT
+ *                                  text-mode translation — NEVER use
+ *                                  _read/_lseeki64 which are racy AND
+ *                                  corrupt 0x0A bytes in binary files).
+ * posix_fadvise -> no-op (advisory only; macOS already no-ops DONTNEED).
+ * posix_memalign->_aligned_malloc(free must be compat_aligned_free).
+ * rename        -> compat_rename (MoveFileEx MOVEFILE_REPLACE_EXISTING;
+ *                                  CRT rename fails EEXIST if dest exists,
+ *                                  breaking stats atomic-write every turn).
+ * meminfo       -> compat_meminfo (GlobalMemoryStatusEx: ullTotalPhys,
+ *                                  ullAvailPhys — approx MemAvailable).
+ * getpid        -> _getpid
+ * =================================================================== */
+#ifdef _WIN32
+
+/* Belt-and-braces: 64-bit off_t mandatory — model is 370 GB, every pread
+ * region can exceed 2 GB. 32-bit off_t silently wraps >4 GB offsets into the
+ * first 4 GB → reads wrong weight bytes → silent token corruption. */
+#if !defined(_FILE_OFFSET_BITS) || _FILE_OFFSET_BITS < 64
+#error "_FILE_OFFSET_BITS=64 required on Windows (add -D_FILE_OFFSET_BITS=64 to CFLAGS)"
+#endif
+
+#ifndef WIN32_LEAN_AND_MEAN
+#define WIN32_LEAN_AND_MEAN
+#endif
+#include <windows.h>
+#include <io.h>
+#include <process.h>
+#include <malloc.h>
+#include <fcntl.h>
+#include <errno.h>
+
+/* --- O_BINARY: belt-and-braces vs CRT text-mode (0x0A byte corruption) --- */
+#ifndef O_BINARY
+#define O_BINARY 0x8000
+#endif
+/* All open() calls for model data must use binary mode.  The compat_pread
+ * wrapper already bypasses CRT via ReadFile on the raw OS handle, so this
+ * is defense-in-depth: if anyone adds a future CRT-based read path, O_BINARY
+ * prevents 0x0A bytes from being silently translated to \r\n. */
+#define COMPAT_O_RDONLY (O_RDONLY | O_BINARY)
+
+/* --- posix_fadvise: no-op (advisory only; safe to ignore) --- */
+#ifndef POSIX_FADV_NORMAL
+#define POSIX_FADV_NORMAL      0
+#define POSIX_FADV_RANDOM      1
+#define POSIX_FADV_SEQUENTIAL  2
+#define POSIX_FADV_WILLNEED    3
+#define POSIX_FADV_DONTNEED    4
+#define POSIX_FADV_NOREUSE     5
+#endif
+#define posix_fadvise(fd,off,len,advice) do{(void)(fd);(void)(off);(void)(len);(void)(advice);}while(0)
+
+/* --- pread -> ReadFile + OVERLAPPED su raw OS handle ---
+ * Thread-safe (no shared seek position). Gestisce offset >4 GB e chunking
+ * per letture >2 GB (anche se i tensori individuali sono nell'ordine dei
+ * MB-centinaia di MB, il wrapper e' robusto per ogni taglia). */
+static inline ssize_t compat_pread(int fd, void *buf, size_t n, off_t off){
+    intptr_t osfh = _get_osfhandle(fd);
+    if(osfh == -1 || osfh == -2){ errno = EBADF; return -1; }
+    HANDLE h = (HANDLE)osfh;
+    size_t total = 0;
+    while(total < n){
+        size_t chunk = n - total;
+        DWORD chunk32 = (chunk > 0x7FFFFFFF) ? 0x7FFFFFFF : (DWORD)chunk;
+        OVERLAPPED ov = {0};
+        ov.Offset     = (DWORD)( (off + (off_t)total)        & 0xFFFFFFFFULL);
+        ov.OffsetHigh = (DWORD)(((off + (off_t)total) >> 32) & 0xFFFFFFFFULL);
+        DWORD rd = 0;
+        if(!ReadFile(h, (char*)buf + total, chunk32, &rd, &ov)){
+            DWORD err = GetLastError();
+            if(err == ERROR_HANDLE_EOF) break;  /* past EOF → return bytes read (0 if none, matching POSIX pread) */
+            if(err == ERROR_INVALID_HANDLE || err == ERROR_INVALID_FUNCTION) errno = EBADF;
+            else errno = EIO;
+            return -1;
+        }
+        total += rd;
+        if(rd == 0 || rd < chunk32) break;  /* EOF or partial (file truncated) */
+    }
+    return (ssize_t)total;
+}
+#define pread(fd,buf,n,off) compat_pread(fd,buf,n,off)
+
+/* --- posix_memalign -> _aligned_malloc ---
+ * ATTN: memoria allocata con _aligned_malloc DEVE essere liberata con
+ * _aligned_free, NON con free(). Vedi compat_aligned_free sotto.
+ * Audit: l'unico sito che libera memoria aligned e' free(s->slab) in
+ * glm.c:892 (cambiato in compat_aligned_free). s->fslab usa falloc()
+ * (malloc semplice) -> il suo free() resta plain. */
+#ifndef ENOMEM
+#define ENOMEM 12
+#endif
+static inline int compat_posix_memalign(void **memptr, size_t alignment, size_t size){
+    if(alignment < sizeof(void*)) alignment = sizeof(void*);
+    *memptr = _aligned_malloc(size, alignment);
+    return *memptr ? 0 : ENOMEM;
+}
+#define posix_memalign(memptr,alignment,size) compat_posix_memalign(memptr,alignment,size)
+
+/* matching free per memoria aligned di _aligned_malloc */
+#define compat_aligned_free _aligned_free
+
+/* --- meminfo: GlobalMemoryStatusEx ---
+ * ullAvailPhys ~ MemAvailable di Linux (include standby/free/zero pages —
+ * pagine recuperabili senza swap). Guida il cap automatico della cache
+ * expert: se sbagliato, la cache e' mis-sized → swap thrash o OOM. */
+static inline void compat_meminfo(double *total_gb, double *avail_gb){
+    MEMORYSTATUSEX msx = {0};
+    msx.dwLength = sizeof(msx);
+    if(GlobalMemoryStatusEx(&msx)){
+        *total_gb = (double)msx.ullTotalPhys / 1e9;
+        *avail_gb = (double)msx.ullAvailPhys  / 1e9;
+    } else {
+        *total_gb = 0; *avail_gb = 0;
+    }
+}
+
+/* --- rename -> MoveFileEx (CRT rename EEXIST se destinazione esiste) ---
+ * stats_dump_q chiama rename(tmp, path) OGNI turno di serve: dopo il primo
+ * write il file esiste gia', e CRT rename fallisce silenziosamente,
+ * affamando la pipeline REPIN/heat/PIN del suo segnale persistente. */
+static inline int compat_rename(const char *old, const char *new){
+    return MoveFileExA(old, new, MOVEFILE_REPLACE_EXISTING) ? 0 : -1;
+}
+#define rename(old,new) compat_rename(old,new)
+
+/* --- getpid -> _getpid --- */
+#define getpid() _getpid()
+
+/* --- rss_gb: getrusage -> GetProcessMemoryInfo ---
+ * ru_maxrss in KB (come Linux): rss_gb() divide per 1e6 → GB corretti. */
+#include <psapi.h>
+#pragma comment(lib, "psapi.lib")
+struct rusage { long ru_maxrss; };
+#define RUSAGE_SELF 0
+static inline int getrusage(int who, struct rusage *r){
+    (void)who;
+    PROCESS_MEMORY_COUNTERS_EX pmc = {0};
+    pmc.cb = sizeof(pmc);
+    if(GetProcessMemoryInfo(GetCurrentProcess(), (PROCESS_MEMORY_COUNTERS*)&pmc, sizeof(pmc))){
+        r->ru_maxrss = (long)(pmc.PeakWorkingSetSize / 1024);  /* ru_maxrss = peak, not current */
+        return 0;
+    }
+    r->ru_maxrss = 0; return -1;
+}
+
+/* --- getline -> compat_getline (fgets + realloc) --- */
+#include <sys/types.h>  /* ssize_t */
+static inline ssize_t compat_getline(char **lineptr, size_t *n, FILE *stream){
+    if(!lineptr || !n || !stream){ errno = EINVAL; return -1; }
+    if(!*lineptr || !*n){ *n = 128; free(*lineptr); *lineptr = malloc(*n); if(!*lineptr) return -1; }
+    size_t pos = 0; int c;
+    while((c = fgetc(stream)) != EOF){
+        if(pos + 1 >= *n){ size_t nn = *n * 2; char *np = realloc(*lineptr, nn); if(!np) return -1; *lineptr = np; *n = nn; }
+        (*lineptr)[pos++] = (char)c;
+        if(c == '\n') break;
+    }
+    if(pos == 0) return -1;
+    (*lineptr)[pos] = '\0';
+    return (ssize_t)pos;
+}
+#define getline(lineptr,n,stream) compat_getline(lineptr,n,stream)
+
+/* --- setenv -> SetEnvironmentVariableA (POSIX setenv assente su Windows) --- */
+static inline int compat_setenv(const char *name, const char *value, int overwrite){
+    if(!overwrite && getenv(name)) return 0;
+    return SetEnvironmentVariableA(name, value) ? 0 : -1;
+}
+#define setenv(name,value,overwrite) compat_setenv(name,value,overwrite)
+
+#endif /* _WIN32 */
+
+/* --- compat_aligned_free su piattaforme diverse da Windows ---
+ * Su Linux/macOS, posix_memalign usa free() normale. */
+#ifndef compat_aligned_free
+#define compat_aligned_free free
+#endif
+
+/* --- COMPAT_O_RDONLY: O_RDONLY con O_BINARY su Windows, O_RDONLY puro altrove --- */
+#ifndef COMPAT_O_RDONLY
+#define COMPAT_O_RDONLY O_RDONLY
+#endif
+
 #endif /* COMPAT_H */
@@ -25,8 +25,8 @@
 #include <limits.h>
 #include <pthread.h>                              /* thread I/O del PILOTA */
 #include <unistd.h>
-#include <sys/resource.h>
 #if defined(__APPLE__) || defined(__linux__)
+#include <sys/resource.h>
 #include <sys/mman.h>                             /* mlock: inchioda le pagine in RAM / wire pages into RAM */
 #endif
 #include "st.h"
@@ -905,7 +905,7 @@ static void expert_load(Model *m, int layer, int eid, ESlot *s){
    /* rialloca se lo slot (riusato tra layer) e' troppo piccolo per QUESTO expert:
     * pread oltre la mappatura = short-read o CORRUZIONE silenziosa dei vicini */
    if(!s->slab || wtot+8192 > s->slab_cap){
-        free(s->slab);
+        compat_aligned_free(s->slab);
        if(posix_memalign((void**)&s->slab,4096,wtot+8192)){fprintf(stderr,"OOM slab\n");exit(1);}
        s->slab_cap=wtot+8192;
    }
@@ -2309,6 +2309,10 @@ static double mem_available_gb(void){
    if(host_statistics64(mach_host_self(),HOST_VM_INFO64,(host_info64_t)&vm,&cnt)!=KERN_SUCCESS) return 0;
    return ((double)vm.free_count+(double)vm.inactive_count+(double)vm.purgeable_count)
           * (double)sysconf(_SC_PAGESIZE) / 1e9;
+#elif defined(_WIN32)
+    double total, avail;
+    compat_meminfo(&total, &avail);
+    return avail;
 #else
    FILE *f=fopen("/proc/meminfo","r"); if(!f) return 0;
    char ln[256]; double kb=0;
@@ -2527,9 +2531,15 @@ int main(int argc, char **argv){
        int *tf=read_arr(ref,"tf_pred",&(int){0});
        int *pred=malloc(nfull*sizeof(int)); double tt=now_s();
        forward_all(&m, full, nfull, pred); double tdt=now_s()-tt;
-        int ok=0; for(int i=0;i<nfull;i++) ok+=(pred[i]==tf[i]);
+        int ok=0; for(int i=0;i<nfull;i++){
+            if(pred[i]==tf[i]) ok++;
+            else fprintf(stderr,"[ORACLE] mismatch pos=%d expected=%d got=%d\n",i,tf[i],pred[i]);
+        }
        printf("PREFILL (teacher-forcing) C vs oracolo: %d/%d posizioni | %.1f pos/s\n",
            ok,nfull,nfull/tdt);
+        if(ok<nfull) fprintf(stderr,
+            "[ORACLE] %d/%d mismatches — run: TF=1 DEBUG_LOGITS=1 for top-5 logit dump\n",
+            nfull-ok,nfull);
        profile_print(&m,tdt);
 #ifdef COLI_CUDA
        if(g_cuda_enabled) cuda_stats_print();
@@ -12,6 +12,7 @@
 #include <time.h>
 #include <errno.h>
 #include <string.h>
+#include "compat.h"
 #ifdef _OPENMP
 #include <omp.h>
 #endif
@@ -12,7 +12,9 @@
 #include <string.h>
 #include <math.h>
 #include <time.h>
+#if defined(__APPLE__) || defined(__linux__)
 #include <sys/resource.h>
+#endif
 #include "st.h"

 /* ---------- config ---------- */
@@ -4,6 +4,7 @@
 import json
 import os
 import re
+import shutil
 import statistics
 import subprocess
 from pathlib import Path
@@ -110,8 +111,11 @@ def build_plan(model, ram_gb=0, context=4096, gpu_indices=None, vram_gb=0,
    cfg = info["config"]
    available_memory = memory_available() if available_memory is None else available_memory
    if available_disk is None:
-        fs = os.statvfs(info["path"])
-        available_disk = fs.f_bavail * fs.f_frsize
+        try:
+            usage = shutil.disk_usage(info["path"])
+            available_disk = usage.free
+        except OSError:
+            available_disk = 500 * GB
    gpus = discover_gpus() if gpus is None else gpus
    if gpu_indices is not None:
        wanted = set(gpu_indices)
@@ -1,24 +1,34 @@
 #!/usr/bin/env bash
-# colibrì — installazione su una macchina nuova (Linux x86-64).
+# colibrì — installazione su una macchina nuova (Linux x86-64, macOS, Windows/MinGW).
 # Compila il motore e fa un self-test. Il MODELLO (~372 GB int4) va copiato a parte
 # o rigenerato con: coli convert --model <dir-su-ext4/NVMe>
 set -e
 cd "$(dirname "$0")"
 echo "🐦 colibrì — setup"

+UNAME_S=$(uname -s)
+
 # 1) dipendenze
 command -v make >/dev/null || { echo "manca make"; exit 1; }
-if [ "$(uname -s)" = "Darwin" ]; then
+case "$UNAME_S" in
+Darwin)
    command -v clang >/dev/null || { echo "manca clang (xcode-select --install)"; exit 1; }
    echo "  clang: $(clang --version | head -1) · $(sysctl -n hw.ncpu) core"
    echo -n "  OpenMP: "
    if brew --prefix libomp >/dev/null 2>&1; then echo "ok (libomp)"
    else echo "libomp assente -> build single-thread (consigliato: brew install libomp)"; fi
-else
+    ;;
+MINGW*|MSYS*)
+    command -v gcc  >/dev/null || { echo "manca gcc (MinGW-w64). Installa: pacman -S mingw-w64-x86_64-gcc make"; exit 1; }
+    echo "  gcc: $(gcc -dumpversion) · MinGW-w64"
+    echo -n "  OpenMP: "; echo 'int main(){return 0;}' | gcc -fopenmp -xc - -o /tmp/_omp 2>/dev/null && echo ok || { echo "manca libgomp (pacman -S mingw-w64-x86_64-gcc)"; exit 1; }
+    ;;
+*)
    command -v gcc  >/dev/null || { echo "manca gcc (es: sudo apt install build-essential)"; exit 1; }
    echo "  gcc: $(gcc -dumpversion) · $(nproc) core"
    echo -n "  OpenMP: "; echo 'int main(){return 0;}' | gcc -fopenmp -xc - -o /tmp/_omp 2>/dev/null && echo ok || { echo "manca (libgomp)"; exit 1; }
-fi
+    ;;
+esac

 # 2) build: nativa (veloce, per QUESTA macchina). Per un binario da distribuire: make portable
 echo "  compilo (ARCH=${ARCH:-native})…"
@@ -31,11 +41,18 @@ if [ -d glm_tiny ] && [ -f ref_glm.json ]; then
 fi

 # 4) info macchina (la velocità dipende da QUESTI due numeri, non dalla GPU)
-if [ "$(uname -s)" = "Darwin" ]; then
+case "$UNAME_S" in
+Darwin)
    ram=$(( $(sysctl -n hw.memsize) / 1000000000 ))
-else
+    ;;
+MINGW*|MSYS*)
+    # MSYS2 fornisce /proc/meminfo come symlink (più affidabile di wmic, deprecato)
    ram=$(awk '/MemTotal/{printf "%.0f", $2/1e6}' /proc/meminfo 2>/dev/null || echo "?")
-fi
+    ;;
+*)
+    ram=$(awk '/MemTotal/{printf "%.0f", $2/1e6}' /proc/meminfo 2>/dev/null || echo "?")
+    ;;
+esac
 echo "  RAM: ${ram} GB   (più RAM = più expert in cache = più veloce)"
 echo
 echo "pronto. Prossimi passi:"
@@ -76,11 +76,11 @@ static inline float f16_to_f32(uint16_t h) {

 static int st_open_fd(shards *S, const char *path) {
    for (int i = 0; i < S->nfd; i++) if (!strcmp(S->paths[i], path)) return S->fds[i];
-    int fd = open(path, O_RDONLY);
+    int fd = open(path, COMPAT_O_RDONLY);
    if (fd < 0) { perror(path); exit(1); }
    S->paths[S->nfd] = strdup(path); S->fds[S->nfd] = fd;
 #ifdef O_DIRECT
-    S->dfds[S->nfd] = open(path, O_RDONLY | O_DIRECT);   /* eager: lookup poi thread-safe */
+    S->dfds[S->nfd] = open(path, COMPAT_O_RDONLY | O_DIRECT);   /* eager: lookup poi thread-safe */
 #elif defined(__APPLE__)
    S->dfds[S->nfd] = compat_open_direct(path);          /* macOS: F_NOCACHE ~ O_DIRECT */
 #else
@@ -0,0 +1,44 @@
+/* audit: verify compat_pread >4GB offset + compat_rename replace-existing (Windows) */
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <fcntl.h>
+#include <sys/stat.h>
+#include <unistd.h>
+#include "../compat.h"
+#include <winioctl.h>
+
+int main(void){
+    /* --- rename second-write test --- */
+    FILE *f = fopen("aud_tmp1", "wb"); fputs("v1", f); fclose(f);
+    if(rename("aud_tmp1", "aud_dst")){ printf("rename #1 FAIL\n"); return 1; }
+    f = fopen("aud_tmp2", "wb"); fputs("v2", f); fclose(f);
+    if(rename("aud_tmp2", "aud_dst")){ printf("rename #2 (dest exists) FAIL <- CRT EEXIST bug\n"); return 1; }
+    char buf[8] = {0};
+    f = fopen("aud_dst", "rb"); fread(buf, 1, 2, f); fclose(f);
+    remove("aud_dst");
+    if(strcmp(buf, "v2")){ printf("rename content FAIL (%s)\n", buf); return 1; }
+    printf("rename replace-existing: ok\n");
+
+    /* --- pread >4GB offset test (sparse file) --- */
+    const char *p = "aud_big.bin";
+    int fd = open(p, O_RDWR | O_CREAT | O_BINARY, 0644);
+    if(fd < 0){ printf("open FAIL\n"); return 1; }
+    /* mark sparse so 5GB costs ~nothing on NTFS */
+    HANDLE h = (HANDLE)_get_osfhandle(fd);
+    DWORD br; DeviceIoControl(h, FSCTL_SET_SPARSE, NULL, 0, NULL, 0, &br, NULL);
+    off_t off = (off_t)5 * 1024 * 1024 * 1024;  /* 5 GiB */
+    const char magic[] = "COLIBRI64";
+    OVERLAPPED ov = {0};
+    ov.Offset = (DWORD)(off & 0xFFFFFFFFULL); ov.OffsetHigh = (DWORD)(off >> 32);
+    if(!WriteFile(h, magic, sizeof(magic), &br, &ov)){ printf("write@5GB FAIL\n"); close(fd); remove(p); return 1; }
+    char rd[sizeof(magic)] = {0};
+    ssize_t n = pread(fd, rd, sizeof(magic), off);
+    close(fd); remove(p);
+    if(n != sizeof(magic) || memcmp(rd, magic, sizeof(magic))){
+        printf("pread >4GB FAIL (n=%lld data=%s) <- OVERLAPPED 64-bit fill broken\n", (long long)n, rd);
+        return 1;
+    }
+    printf("pread >4GB offset: ok\n");
+    return 0;
+}
@@ -0,0 +1,115 @@
+#!/usr/bin/env python3
+"""Convert OLMoE HuggingFace checkpoint to colibri int4 format.
+
+Downloads or converts a local OLMoE checkpoint (e.g., allenai/OLMoE-1B-7B-0125-Instruct).
+Dense weights stay as-is (engine reads BF16/F16 → F32 on load).
+Expert weights get row-wise int8 quantization with float32 scales.
+
+Usage:
+  python tools/convert_olmoe.py --repo allenai/OLMoE-1B-7B-0125-Instruct --out ./olmoe_i4
+  python tools/convert_olmoe.py --model ./OLMoE-1B-7B-0125-Instruct --out ./olmoe_i4
+"""
+
+import argparse, json, math, os, struct, sys
+from pathlib import Path
+
+# Windows: force UTF-8 output
+if sys.platform == "win32":
+    for s in (sys.stdout, sys.stderr):
+        try: s.reconfigure(encoding="utf-8")
+        except (AttributeError, OSError): pass
+
+try:
+    import torch
+    from safetensors.torch import load_file, save_file
+except ImportError as exc:
+    sys.exit(f"Missing dependencies: {exc}. Install: pip install torch safetensors")
+
+
+EXPERT_KEY_RE = r"model\.layers\.\d+\.mlp\.experts\.\d+\.(gate_proj|up_proj|down_proj)\.weight"
+
+
+def quantize_row(w: torch.Tensor) -> tuple[torch.Tensor, torch.Tensor]:
+    """Row-wise int8 quantization. Returns (int8_weights, float32_scales)."""
+    w_f32 = w.float()
+    row_max = w_f32.abs().amax(dim=1, keepdim=True).clamp(min=1e-12)
+    scales = row_max / 127.0
+    q = (w_f32 / scales).round().clamp(-128, 127).to(torch.int8)
+    return q, scales.squeeze(1)
+
+
+def is_expert_weight(name: str) -> bool:
+    import re
+    return bool(re.search(EXPERT_KEY_RE, name))
+
+
+def main():
+    ap = argparse.ArgumentParser(description="Convert OLMoE HF checkpoint -> colibri int4")
+    src = ap.add_mutually_exclusive_group(required=True)
+    src.add_argument("--repo", help="HuggingFace repo ID")
+    src.add_argument("--model", help="Local HF checkpoint directory")
+    ap.add_argument("--out", required=True, help="Output directory for int4 model")
+    ap.add_argument("--ebits", type=int, default=4,
+                    help="Expert quant bits (4 or 8, default 4)")
+    args = ap.parse_args()
+
+    if args.repo:
+        from huggingface_hub import snapshot_download
+        print(f"Downloading {args.repo}...")
+        src_dir = snapshot_download(args.repo, local_files_only=True, max_workers=4)
+        if not any(Path(src_dir).glob("*.safetensors")):
+            print("Downloading safetensors...")
+            src_dir = snapshot_download(args.repo, max_workers=4)
+    else:
+        src_dir = args.model
+
+    src = Path(src_dir)
+    if not src.is_dir():
+        sys.exit(f"Model directory not found: {src}")
+    if not (src / "config.json").is_file():
+        sys.exit(f"config.json missing in {src}")
+
+    out = Path(args.out)
+    out.mkdir(parents=True, exist_ok=True)
+
+    # Copy config.json
+    import shutil
+    shutil.copy2(src / "config.json", out / "config.json")
+    print(f"config.json -> {out}")
+
+    # Process safetensors
+    shards = sorted(src.glob("*.safetensors"))
+    if not shards:
+        sys.exit(f"No safetensors found in {src}")
+
+    expert_count = 0
+    total_expert_f32 = total_expert_q = 0
+
+    for si, shard in enumerate(shards, 1):
+        print(f"[{si}/{len(shards)}] {shard.name}...", end=" ", flush=True)
+        tensors = load_file(str(shard))
+        out_tensors = {}
+        for name, tensor in tensors.items():
+            if is_expert_weight(name):
+                expert_count += 1
+                q, scales = quantize_row(tensor)
+                total_expert_f32 += tensor.numel() * tensor.element_size()
+                total_expert_q += q.numel() * 1 + scales.numel() * 4
+                out_tensors[name] = q
+                out_tensors[name + ".qs"] = scales
+            else:
+                out_tensors[name] = tensor
+
+        out_shard = out / shard.name
+        save_file(out_tensors, str(out_shard))
+        ratio = total_expert_q / max(total_expert_f32, 1) * 100
+        print(f"ok")
+
+    print(f"\nDone. {expert_count} expert tensors quantized.")
+    print(f"Expert storage: {total_expert_f32/1e9:.1f} GB -> {total_expert_q/1e9:.1f} GB ({ratio:.0f}%)")
+    print(f"Model ready at: {out}")
+    print(f"\nRun: SNAP={out} ./olmoe.exe 32 4 16")
+
+
+if __name__ == "__main__":
+    main()