KV-to-disk: conversations reopen WARM across engine restarts (.coli_kv, KVSAVE=0 opts out)
serve mode persists the compressed MLA KV-cache incrementally after every turn (~182 KB/token appended, header count written last = crash-safe) and resumes it at startup: the model remembers the whole conversation and zero re-prefill happens. :reset and context-full restarts truncate the file. The MTP layer's KV row is not saved; kv_start=-1 re-arms its decode window. Validated: split-session answer byte-identical to an uninterrupted session (tiny oracle, TEMP=0), and on the real 744B model a restarted chat resumed 58 tokens in 0.0s and recalled a fact from the previous session while prefilling only the new question. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
@@ -289,7 +289,7 @@ def cmd_chat(a):
|
||||
mload=re.search(r"caricato in ([0-9.]+)s \| densa residente: ([0-9.]+) MB", elog)
|
||||
if mload: print(f" {C.grn}✓{C.r} pronto in {mload.group(1)}s {C.dim}· residente {float(mload.group(2))/1000:.1f} GB · RSS {st.get('rss','?')} GB{C.r}")
|
||||
for l in elog.splitlines(): # una riga di stato per riga, senza path
|
||||
if l.startswith(("[RAM_GB","[PIN]","[MTP]","[USAGE]","[DSA]")):
|
||||
if l.startswith(("[RAM_GB","[PIN]","[MTP]","[USAGE]","[DSA]","[KV]")):
|
||||
l=re.sub(r" ?\(?/[^ )]+\)?","",l.strip()) # via i percorsi lunghi
|
||||
l=re.sub(r" da$| in$","",l)
|
||||
for chunk in textwrap.wrap(l, term_w()-4) or [l]:
|
||||
|
||||
Reference in New Issue
Block a user