hohne.meWelcome Hohne

Writing

What a compacting memory forgets, counted by cause

Long agent conversations outgrow the context window, so many systems compact them. Older messages get summarized, and the agent works from the summaries. I wanted to know which facts survive that process and whether the model can get a lost detail back when it needs it. I also wanted every failure labeled with a cause, because a single recall score does not tell you what to change. This page describes the test I built, what it showed, and what I would tell a team building memory for an agent.

The system under test is UniiChat. It compacts a long chat into a tree of summary lines, each at most 512 bytes.

The question

The chat model sees a view made of some of those lines. When a line looks relevant, the model can zoom, which means opening the line to see the more detailed lines or messages under it. The question was which facts survive compaction, and whether zooming recovers the ones the summaries dropped.

A score of 15 out of 20 gives no hint of what to fix. The same score with every failure labeled "lost silently" points to a different fix than it does with every failure labeled "stale." So the cause labels were part of the design from the start.

The setup

The test chats are synthetic and seeded, one with 128 messages and one with 256. Each contains planted facts of seven types: code, correction, date, decision, name, negative, and number. A correction is a value that changes later in the chat, so a model that returns the earlier value has failed in a specific way. Each planted fact gets a probe question several turns after it appears.

Every failed probe gets exactly one of six causes:

The harness ran on a MacBook Pro M3 Pro with 36 GB of memory. The chat model and the gemma and deepseek compactors were Ollama cloud models, gemma4:31b-cloud and deepseek-v4.1-flash:cloud, on Ollama 0.40.2. The runs were on October 11 and October 12, 2026. Each 128 message run used 20 probes, and each 256 message run used 41 probes. Every score on this page is a single run. The 18 of 20 result is seed 7. I treat a gap of one or two probes on a 20 probe run as noise. Prompts are frozen, and every run checks them for drift. Each run logs the model quantization, the Ollama version, the drift check, and the bytes, seconds, and calls per probe. A script, rescore.py, recomputes every metric from the saved tree and call log without rerunning a model. There are 41 recorded runs. I left 6 smoke tests out of that count.

What I found

The compactor matters, and gemma4:31b-cloud did best

The compactor is the model that writes the summary lines. On the 128 message chat, seed 7, gemma4:31b-cloud scored 18 of 20, deepseek-v4.1-flash:cloud scored 16 of 20, and glm-5.3-flash scored 14 of 20. glm-5.3-flash took 73 minutes, and I dropped it. On seed 11, gemma4:31b-cloud scored 16 of 20 and deepseek-v4.1-flash:cloud scored 10 of 20. Each of these is a single run. gemma4:31b-cloud led on both seeds. The seed 7 gap is two probes, which I treat as noise, and the seed 11 gap is six probes, so I would not rank the compactors from seed 7 alone.

A bigger view mostly bought bytes

The view budget is how much summary text the chat model sees at once. I doubled it from 8 KB to 16 KB on the 256 message chat. For gemma4:31b-cloud, seed 11 moved from 17 to 29 of 41, a gain of 12. Seed 7 moved from 27 to 25, a loss of 2. Bytes per probe doubled, from about 7 KB to about 14 KB. Zooms per probe fell from between 0.3 and 1.4 to between 0.1 and 0.4.

The cause labels showed what changed. With more text in view, the model stopped zooming, and failures shifted toward "lost silently." My reading is that it answered from summary lines that had already dropped the detail. Without the labels, I would have seen only a mixed recall number.

The 256 message chat

At an 8 KB budget, recall out of 41 depended on the line length and the seed.

Line length gemma4:31b-cloud, seed 7 deepseek-v4.1-flash:cloud, seed 7 gemma4:31b-cloud, seed 11 deepseek-v4.1-flash:cloud, seed 11
512 bytes 27 17 17 23
768 bytes 28 25 27 17
1024 bytes 27 19 29 26

tev1:4b as the router scored 21 of 41 on seed 7 and 25 of 41 on seed 11. Those router runs used gemma4:31b-cloud as the compactor. I have no compactor comparison with the router on. Numbers were the weakest fact type, at 15 of 41.

A small router removed silent losses and created wrong branches

Next I put tev1:4b in front of the chat model as a navigator. A decision model returns a probability for each of a fixed set of options instead of writing text. Its job here was to pick which summary line to open. On recorded trees, tev1:4b chose correctly 84 to 92 percent of the time per decision, against 49 percent for random choice.

Live, end to end, the result was flat. The system scored 79 of 122 with tev1:4b and 78 of 122 without it, a one probe difference inside my noise band.

Underneath that flat total, the causes moved a lot. Those counts are on the same 122 probes. Without the router there were 44 failures: 29 lost silently, 6 wrong branch, and 9 other. With tev1:4b there were 43 failures: 0 lost silently, 30 wrong branch, and 13 other. The router removed the silent losses, and once it was on the right branch it followed that branch down well. The failure was the first choice, which meant picking which of 20 view lines held the answer. When that first pick was right, recall was 93 percent. When it was wrong, recall was 61 percent. The first pick was right only 27 percent of the time. I take this to mean that per decision accuracy on recorded trees overstates live performance when the first choice decides everything after it.

clef-flash ran only on the 128 message chats, where it scored 18 of 20 and 15 of 20. It is the 9B model in that document sorting test.

A logging bug multiplied the compaction work

At first I logged the router's zooms into the chat log. Both the broken run and the fixed run used a 128 message chat, seed 7, gemma4:31b-cloud as the chat model and the compactor, tev1:4b as the router, 512 byte lines, and an 8192/4096 budget. Only the router logging changed. The log grew from 128 lines to 406 because the zooms were written into it, and the router followed its own echoes. The buggy run made 655 compaction calls, against 204 with no router and 88 after the fix. Extra compaction from the decision model adds noise. Keeping the router's zooms out of the tree avoids that. The fix was to hand the opened text to the chat model without writing it into the log. The first run with the fix scored 19 of 20 in 13 minutes, against 15 of 20 in 93 minutes before the fix.

What I would tell a team building agent memory

Label every recall failure with a cause before you tune anything. Without the labels you will tune the wrong setting. In my runs the view budget looked like a recall setting. The labels showed that it mostly changed how often the model looked things up.

Measure bytes per answer separately from recall. Doubling the view doubled the bytes and did not reliably improve recall.

Expect new failures at every handoff. Adding a second model to navigate removed 29 silent losses and added 24 wrong branch failures. Score a router live, end to end, as well as on recorded data, and look at where the failures moved.

Keep decision models out of the tree that gets compacted. Extra compaction work from those models leads to noise. My logging bug let the router read its own earlier zooms as if they were part of the chat. Passing the opened text straight to the chat model fixed it.

Save what you need to rescore. Because each run keeps its tree and call log, I can change a scoring rule and recompute the metrics without running the models again. Record the exact model artifact too. On Evaluation I explain why a model name is not a specification.

Limits

The chats are synthetic, with 128 and 256 messages and seven types of planted fact. Real conversations are messier, and I have not shown that these numbers carry over to them. Each score is a single run, and seed 7 and seed 11 do not always agree.

Sources