 ZacharyZcR
|
9e6e1ac327
|
Isolated sequence KV contexts: KVState extraction, KV_SLOTS serve contexts with per-slot persistence, budget-aware pool accounting (#29)
* Add isolated sequence KV contexts
* Log projected multi-slot KV footprint
|
2026-07-10 16:19:43 +02:00 |
|
|
ZacharyZcR
|
76e858a80f
|
Bounded FIFO inference scheduling: admission queue, OpenAI-shaped 429/503, queued-disconnect detection (#28)
|
2026-07-10 12:52:07 +02:00 |
|
|
ZacharyZcR
|
3e4d08b6bf
|
OpenAI-compatible HTTP API: stdlib-only gateway over SERVE with KV prefix reuse across stateless requests (#21)
* Add OpenAI-compatible HTTP API
* Support browser API clients
* Handle missing KV cache during rewind
|
2026-07-10 11:04:56 +02:00 |
|