Organize project tools and local workflows: c/tools, c/scripts, c/tests, root Makefile (flat C core untouched) (#22)

This commit is contained in:
ZacharyZcR
2026-07-10 16:08:39 +08:00
committed by GitHub
parent a2942b2172
commit 13e8f09ffc
23 changed files with 87 additions and 52 deletions
+21 -11
View File
@@ -36,7 +36,7 @@ The engine is a single C file (`c/glm.c`, ~1,300 lines) plus small headers. No B
- **Batch-union MoE**: in prefill (and MTP verification), each unique expert of the batch is read once and applied to every position that routes to it.
- **Byte-level BPE tokenizer in C** (GPT-2-style with Unicode-property regex, 320k merges).
- **RAM safety**: the expert cache is auto-sized from `MemAvailable` at startup — an honest peak projection (working set, KV, MTP row, reconstruction buffers) so the kernel OOM-killer never fires.
- **Offline FP8→int4 converter** (`c/convert_fp8_to_int4.py`): downloads one shard at a time (~5 GB), dequants (128×128 block scales), requantizes to the engine's container, deletes the shard — the 756 GB FP8 checkpoint never needs to exist on disk at once. Resumable.
- **Offline FP8→int4 converter** (`c/tools/convert_fp8_to_int4.py`): downloads one shard at a time (~5 GB), dequants (128×128 block scales), requantizes to the engine's container, deletes the shard — the 756 GB FP8 checkpoint never needs to exist on disk at once. Resumable.
## Honest numbers (WSL2, 12 cores, 25 GB RAM, NVMe via VHDX)
@@ -147,8 +147,8 @@ deterministic 313M-parameter `glm_moe_dsa` fixture and run fixed-token replay:
```bash
cd c
python make_glm_bench_model.py --output /nvme/colibri-bench-medium --device cuda
python benchmark_cuda_fixture.py --model /nvme/colibri-bench-medium --gpu 0
python tools/make_glm_bench_model.py --output /nvme/colibri-bench-medium --device cuda
python tools/benchmark_cuda_fixture.py --model /nvme/colibri-bench-medium --gpu 0
```
The fixture has random weights and is not a language model. It exists only to
@@ -242,16 +242,26 @@ Every contribution, from a datapoint to a disk, moves the ceiling.
## Repo layout
```
c/glm.c the engine (GLM-5.2 forward, streaming MoE, MTP, serve mode)
c/st.h safetensors reader: pread + fadvise, no mmap (RSS stays flat)
c/tok.h byte-level BPE tokenizer in C
c/coli CLI: chat / run / bench / convert / info
c/iobench.c parallel disk microbenchmark (measures what the engine feels)
c/convert_fp8_to_int4.py disk-safe FP8 → int4 converter
c/make_glm_oracle.py tiny-random oracle generator for validation
c/olmoe.c stage-A engine (OLMoE), first validation target
Makefile root build/check entry point
c/
├── glm.c single-file GLM engine
├── st.h, tok.h, json.h runtime headers
├── backend_cuda.* optional CUDA tier
├── Makefile build and local checks
├── coli user-facing CLI
├── setup.sh one-command local setup
├── tools/ offline conversion, fixtures and benchmarks
├── scripts/ long-running conversion helpers
└── tests/ dependency-free C and Python tests
```
The runtime path intentionally stays flat and readable: `glm.c` plus its small
headers. Auxiliary Python and shell tooling is grouped separately and is never a
runtime dependency of the engine.
From the repository root, `make`, `make check`, and `make clean` delegate to the
engine Makefile. Existing commands run from `c/` continue to work unchanged.
## Why "colibrì"
The hummingbird weighs a few grams, hovers in place, and visits a thousand flowers a day. This engine keeps a 744-billion-parameter giant alive on hummingbird rations: 25 GB of RAM, twelve CPU cores, and a lot of disk patience.