The brain finally got big enough to start fooling itself, and I have the numbers.
The question I’ve been circling since spring: as a personal memory corpus grows, does retrieval get worse? The comfortable assumption is that embeddings plus vector search just works — ask a question, get the right memory back. At a few hundred notes, it does. The interesting part is what happens after that, and until June it was a hypothesis. Now it’s a measurement.
What the margin is
For any question you ask a memory, there are two documents that matter: the right answer, and the most convincing wrong one. The margin is the gap between them — how much more the right memory resembles your question than the best imposter does. Formally it’s Δ(q) = cos(q, right) − max cos(q, distractor), but the intuition is enough: wide margin, the system can’t miss; margin near zero, “search” becomes a coin flip wearing a confidence score.
The measurement
On June 24 I ran the first margin measurement over the real corpus — the canonical notes collection, 2,884 notes that day (the curated layer retrieval actually searches, not the raw telemetry stream). Fifteen hand-written probe questions with 39 known-correct answers, every one verified present in the corpus before the run. Then slices: 500 notes, 1,000, 2,000, 2,749 — built so the right answers always stay in the pool. Nothing is ever “lost.” The only thing that changes is how many plausible strangers crowd in around the truth.
| corpus slice | mean margin | right answer wins outright |
|---|---|---|
| 500 | 0.090 | 93% |
| 1,000 | 0.051 | 80% |
| 2,000 | 0.011 | 60% |
| 2,749 | 0.023 | 67% |
The margin closes: 0.090 → 0.051 → 0.011. And the mechanism is almost embarrassingly simple once you see it. The right answer’s resemblance to the question is fixed — it’s a property of the match. But the best distractor is a maximum over the crowd, and maxima only go up. Every new capture is another lottery ticket for “most convincing wrong answer.” At 2,000 notes, the right memory out-scores every stranger only 60% of the time.
Honest tail, logged as measured: the 2,749-note point ticks back up (0.011 → 0.023, 60% → 67%). Not cleanly monotonic. With fifteen probes that’s most likely small-sample noise, possibly an early plateau — the 500→2,000 trend is unambiguous either way. No smoothing. This notebook doesn’t do that.
Why “conservation”
We tried the obvious lever: prefilter by type and domain before searching, so the question only competes inside its own neighborhood. It works — the whole curve lifts, better at every corpus size. And the lifted curve still declines. The lever buys headroom; the geometry keeps closing underneath it. That’s why I’ve been calling it a conservation law: you can move the collapse, you can’t cancel it.
Why I care
This is the fidelity problem made measurable — the actual reason a small local model gets confidently wrong as its memory grows. It isn’t that the model degrades. It’s that the ground it stands on gets more crowded, and cosine similarity stops being able to tell the witness from the lookalikes.
The useful twist is that the number diagnosing the disease is also the treatment. That win-rate column is a confidence signal: when the margin says not sure, the honest move is to abstain and say so — which is exactly the behavior you want from a gate deciding what leaves the node. A system that knows when its own memory is ambiguous is worth more than one that answers everything.
Measure on the way up
One more property of this curve, and it’s the one that makes me glad I ran it in June: the left side is non-recapturable. Once a corpus grows past a given size, you can never again measure what retrieval felt like when it was small. Every growing memory passes each N exactly once.
So if you’re running your own corpus — and the whole thesis of this site is that you should be — measure early, measure often. The low-N points are data you can only collect on the way up.
The method, harness, and raw run logs live in the research files; this is the notebook version. The margin gets re-measured as the corpus grows — that trajectory is the actual experiment.