olmoe: reject quant_bits outside 2..8 (fixes degenerate output, #134) + correct ref.json to OLMoE-0125-Instruct oracle (#133)

#134: olmoe.c stored experts as int8_t but silently accepted any bits argv;
bits=16 (falsely documented as f32) truncated in quantize_rows -> wraparound
garbage experts. Guard bits to 2..8 with a clear error (int8 is token-exact;
f32 experts are not implemented here, unlike glm.c's fmt=0 path). Doc corrected.

#133: shipped ref.json was from a different checkpoint (continued 'The capital
of the United States is Washington'); the intended target is
allenai/OLMoE-1B-7B-0125-Instruct (per convert_olmoe.py), whose greedy oracle
continues 'The official language of France is French'. full_ids/text updated to
the reporter's verified oracle (engine is already token-exact 12/12 vs it).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
JustVugg
2026-07-13 20:33:47 +02:00
parent 4418e16db4
commit 1f0e1b7076
2 changed files with 35 additions and 2 deletions
+30 -1
View File
@@ -1 +1,30 @@
{"prompt": "The capital of France is", "prompt_ids": [510, 5347, 273, 6181, 310], "full_ids": [510, 5347, 273, 6181, 310, 7785, 15, 187, 187, 510, 5347, 273, 253, 1986, 2077, 310, 5041], "text": "The capital of France is Paris.\n\nThe capital of the United States is Washington"}
{
"prompt": "The capital of France is",
"prompt_ids": [
510,
5347,
273,
6181,
310
],
"full_ids": [
510,
5347,
273,
6181,
310,
7785,
15,
187,
187,
510,
3565,
3448,
273,
6181,
310,
5112,
15
],
"text": "The capital of France is Paris.\n\nThe official language of France is French."
}