olmoe: reject quant_bits outside 2..8 (fixes degenerate output, #134) + correct ref.json to OLMoE-0125-Instruct oracle (#133)

#134: olmoe.c stored experts as int8_t but silently accepted any bits argv;
bits=16 (falsely documented as f32) truncated in quantize_rows -> wraparound
garbage experts. Guard bits to 2..8 with a clear error (int8 is token-exact;
f32 experts are not implemented here, unlike glm.c's fmt=0 path). Doc corrected.

#133: shipped ref.json was from a different checkpoint (continued 'The capital
of the United States is Washington'); the intended target is
allenai/OLMoE-1B-7B-0125-Instruct (per convert_olmoe.py), whose greedy oracle
continues 'The official language of France is French'. full_ids/text updated to
the reporter's verified oracle (engine is already token-exact 12/12 vs it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
JustVugg
2026-07-13 20:33:47 +02:00
parent 4418e16db4
commit 1f0e1b7076
2 changed files with 35 additions and 2 deletions
+5 -1
View File
@@ -39,7 +39,7 @@ typedef struct { Slot *slots; int n, cap; } LCache;
typedef struct {
Cfg c;
shards S;
int quant_bits; /* bit di quantizzazione degli expert (2..8); 16 = f32 */
int quant_bits; /* bit di quantizzazione degli expert (2..8); storage int8, niente f32 (#134) */
float *embed, *lm_head, *final_norm;
Layer *L;
LCache *cache; /* [n_layers] */
@@ -360,6 +360,10 @@ int main(int argc, char **argv) {
if (!snap) { fprintf(stderr, "set SNAP=<snapshot directory>\n"); return 1; }
int cap = argc > 1 ? atoi(argv[1]) : 16;
int bits = argc > 2 ? atoi(argv[2]) : 8;
if (bits < 2 || bits > 8) { /* expert storage is int8_t: bits>8 truncates in quantize_rows (#134). f32 mode is not implemented here — int8 is already token-exact vs the oracle. */
fprintf(stderr, "quant_bits must be 2..8 (got %d); OLMoE experts are int8-backed, no f32 mode\n", bits);
return 1;
}
const char *refpath = argc > 3 ? argv[3] : "ref.json";
FILE *f = fopen(refpath, "rb"); if(!f){perror(refpath);return 1;}