iobench: Windows correctness — aligned free, 64-bit totals, full-range random offsets (also improves Linux offset coverage) (#64)
* fix: Windows port audit fixes — _FILE_OFFSET_BITS guard, O_BINARY st.h, getrusage peak, oracle diagnostic, setup.sh wmic
Audit remediation (all MEDIUM issues fixed):
- compat.h: compile-time #error if _FILE_OFFSET_BITS < 64 on _WIN32
- compat.h: COMPAT_O_RDONLY macro (O_RDONLY|O_BINARY on Windows, belt-and-braces)
- st.h: use COMPAT_O_RDONLY in both open() call sites (plan §1 row 1)
- compat.h: getrusage shim now uses PeakWorkingSetSize (ru_maxrss = peak, not current)
- glm.c: oracle mismatch diagnostic — prints position/expected/got on TF failures
- setup.sh: replace deprecated wmic with /proc/meminfo (MSYS2 provides it)
- .gitignore: *.exe, glm_tiny/, olmoe_hf/, olmoe_i4/
LOW issues addressed:
- _FILE_OFFSET_BITS guard prevents silent 32-bit off_t wrap at >4GB offsets
- coli Windows venv path (Scripts/python.exe) fixed earlier
- posix_fadvise do{}while(0) kept intentionally — no caller checks return code
Verification: oracle 32/32, 27/27 Python tests, rename EEXIST, >4GB pread offset.
* docs: add Windows 11 native port section to README
- Toolchain: MinGW-w64 (winlibs or MSYS2), GCC 16.1 tested
- Build instructions: make glm.exe, tiny oracle verification
- Runtime: SNAP=..., coli chat, coli serve all work
- Status: Phase 1 complete (compiles, correct, static-linked)
- Update platform requirements to include Windows 11 natively
* fix: iobench Windows correctness — aligned free, 64-bit totals, full-range offsets
Three bugs found running iobench.exe on Windows 11 (MinGW-w64):
1. Heap corruption (0xC0000374): free() on posix_memalign'd buffer.
On Windows posix_memalign maps to _aligned_malloc, whose memory must
be released with _aligned_free. Use compat_aligned_free (plain free
on POSIX, _aligned_free on Windows).
2. Negative GB totals: 'long tot' is 32-bit on Windows (LLP64), so any
benchmark reading more than 2 GB overflowed. Now int64_t.
3. Inflated speeds: RAND_MAX is 32767 on Windows, so rand()*4096 only
generated offsets in the first 134 MB of the file — which the page
cache holds entirely after one pass, reporting RAM speed instead of
disk speed (measured 6 GB/s fake vs 2.1 GB/s real on a laptop NVMe).
Combine two rand() calls into 30 bits so offsets span the whole file.
Portable: same code path on Linux/macOS, same seeded sequence shape.
This commit is contained in:
+6
-4
@@ -38,10 +38,12 @@ int main(int argc,char**argv){
|
||||
if(fd<0){perror("open");return 1;}
|
||||
off_t sz=lseek(fd,0,SEEK_END);
|
||||
if(sz<blk*2){fprintf(stderr,"file troppo piccolo\n");return 1;}
|
||||
/* offset random pre-generati (stessi per ogni configurazione: srand fisso) */
|
||||
/* offset random pre-generati (stessi per ogni configurazione: srand fisso).
|
||||
* 30 bit di rand combinati: su Windows RAND_MAX=32767 e un singolo rand()*4096
|
||||
* copre solo i primi 134 MB del file (tutti in page cache = misura falsa). */
|
||||
off_t *offs=malloc(n*sizeof(off_t)); srand(1234);
|
||||
for(int i=0;i<n;i++){ off_t o=((off_t)rand()*4096)%(sz-blk); offs[i]=o&~4095L; }
|
||||
double t0=now(); long tot=0;
|
||||
for(int i=0;i<n;i++){ off_t r30=((off_t)rand()<<15)|rand(); off_t o=(r30*4096)%(sz-blk); offs[i]=o&~4095L; }
|
||||
double t0=now(); int64_t tot=0; /* long e' 32-bit su Windows (LLP64): >2GB andava in overflow */
|
||||
#pragma omp parallel num_threads(nth) reduction(+:tot)
|
||||
{
|
||||
void *buf; if(posix_memalign(&buf,4096,blk)){perror("memalign");exit(1);}
|
||||
@@ -50,7 +52,7 @@ int main(int argc,char**argv){
|
||||
ssize_t r=pread(fd,buf,blk,offs[i]);
|
||||
if(r<0) perror("pread"); else tot+=r;
|
||||
}
|
||||
free(buf);
|
||||
compat_aligned_free(buf); /* su Windows posix_memalign=_aligned_malloc: free() corrompe l'heap */
|
||||
}
|
||||
double dt=now()-t0;
|
||||
printf("%s x%d thread: %d letture x %ldMB = %.1f GB in %.2fs -> %.2f GB/s (%.1f ms/blocco effettivi)\n",
|
||||
|
||||
Reference in New Issue
Block a user