Context engineering: clearing the noise
1. Introduction
When a coding agent’s context window fills up, it compacts: it replaces the older part of the conversation with a summary and keeps going. Every major agent does it, and the people who use these agents most don’t trust it.
On r/ClaudeCode, a thread asking “Do you tell Claude before running /compact?” drew almost a hundred comments from people who prepare the model before compacting, in case it drops something. Another user, running Claude Code mostly on its own, names compaction as one of the two walls that stop it: it “gets auto-compacted mid-edit and loses the thread.” On r/codex, users report agents “forgetting the tasks they were on […] like finding a bug, compacting, then skipping actually fixing the bug.”
So people build their own fixes: plugins that compact earlier, write a checkpoint before compacting, or compact when a task is done. One user replayed 122 compaction points from their own sessions to test a popular compaction plugin before installing it, and found that only about 1% of the tool output their sessions later needed survived.
The complaints come down to three things. Compaction loses details the agent needs later. It happens at the wrong moment, in the middle of a task. And it is expensive: until it fires, every request re-sends the whole history; on a 1M-token model, Claude Code waits until about 967k tokens.
This post looks at why, through the lens of context engineering: what actually fills an agent’s context (section 2), a compaction design built around a full session log (section 3), and a test of how well different compaction setups keep a detail that is needed later (section 4).
2. Background
2.1 Long contexts degrade
Context engineering is the work of deciding what goes into a model’s context window at each step: instructions, files, tool results, history. Andrej Karpathy put it as “the delicate art and science of filling the context window with just the right information for the next step.” Too little, and the model guesses. Too much, and it gets slower, more expensive and worse.
“Too much” comes sooner than the size of the window suggests. A 128k window doesn’t mean 128k tokens of useful attention, and three lines of research show where it breaks down.
Models get worse as the input grows, and newer models only push the drop further out. In 2025, NoLiMa found GPT-4o falling from 99% on short inputs to 70% at 32k tokens.Modarressi et al., NoLiMa: Long-Context Evaluation Beyond Literal Matching, ICML 2025. Current models last much longer, but the pattern holds. ATLAS, a May 2026 benchmark of 26 models from 8k to 1M tokens, scores every model lower once lengths up to 1M are included than on lengths up to 128k:
The drop depends a lot on the model and the task. Claude Opus 4.6 lost the least (8.5%). For simple retrieval, Gemini 3.1 Pro fell from 85 at 128k to 50 at 1M.Huang et al., ATLAS: All-round Testing of Long-context Abilities across Scales, 2026 (arXiv 2605.28079). On its own 8-needle test at 1M tokens, Anthropic reports 76% for Opus 4.6 against 18.5% for Sonnet 4.5 (February 2026).
Where the information sits matters. In Lost in the Middle (2023), GPT-3.5 answered correctly 76% of the time when the relevant document came first among 20, and 54% when it sat in the middle, worse than with no documents at all.Liu et al., Lost in the Middle: How Language Models Use Long Contexts, TACL 2023. Tables 1 and 6.
Irrelevant context hurts even when the answer is there. Chroma ran 18 models on a memory benchmark twice: once with the full chat history (about 113k tokens) and once with only the relevant part (about 300 tokens). Every model scored significantly higher with the short version.Hong, Troynikov and Huber, Context Rot: How Increasing Input Tokens Impacts LLM Performance, Chroma, 2025.
So a context window works less like storage and more like attention: every token you add takes a little away from all the others.
For a chatbot, this is mostly about the prompt. For an agent, it’s about everything that piles up while it works: files it read, commands it ran, errors, its own reasoning. An agent writes most of its own context, and that’s where things go wrong.
2.2 Where the tokens go
Before designing anything, I measured what fills a real session. I took six of my own Claude Code sessions, about 15 million tokens in total, and counted each kind of content.
Tool outputs dominate: file reads, command results, search results. My own messages, the part that feels like “the conversation”, are 2%.
That changed how I think about the problem. Context engineering for an agent is mostly about tools: what they return, how long it stays, and what replaces it later.
2.3 Why compaction is hard
Compaction has to decide what the rest of the session will need, before anyone knows. Decisions survive a summary well, because they are short and stated outright. What tends to disappear is what was tried and dropped, and why: lose that, and the next stretch of work can rebuild the approach that already failed. That’s why I say compaction without memory is just a new session.
The cheapest fix doesn’t go far enough either. In my measurement, clearing every old tool output makes a session only about 2.5x smaller, and the session keeps growing.
2.4 How agents compact today
I read the compaction code of five open-source agents. They make quite different choices.
| Agent | When | What it keeps | What it summarizes |
|---|---|---|---|
| Codex | a token limit | the user’s messages, up to 20k tokens | a handoff summary for “another model” |
| OpenCode | window minus 20k | the newest 40k tokens of tool output; older ones cleared first | the rest, with an LLM |
| pi | window minus 16k | the last 20k tokens, cut at turn boundaries | a structured summary, updated at the next compaction rather than rewritten |
| Gemini CLI | 50% of the window | the newest 30% | a snapshot, then a second pass that asks “did you omit any paths, errors, constraints?” |
| OpenHands | over 240 events | the first 2 events and the newest half | a rolling summary |
Codex’s choice is the one I disagree with most. Keeping only the user’s messages means the agent forgets everything it learned: which files matter, which commands failed, what the code actually says. Users report agents redoing tasks they had already finished.
2.5 Memory outside the window
The teams behind these agents know compaction loses things. Over the past year, each added a memory outside the context window, in files the agent writes and reads back: Anthropic’s memory tool and Claude Code’s CLAUDE.md and auto memory, Codex’s Memories, and Gemini CLI’s layered Markdown memory. Claude Code’s docs even give the fix for an instruction lost after /compact: put it in CLAUDE.md.
The direction is clear: what matters must live outside the window. And all of these work the same way: the agent decides, as it works, what is worth writing down.
That works well for preferences, decisions and facts about the project. It works less well for details, because choosing what to save means predicting what will matter later. An agent reading a config file has no reason to save “the retryable error codes are E2033, E3201 and E5179” until someone asks, and by then the file may be gone from the window.
So my design does both. The agent writes notes for what it knows matters (decisions, rejected approaches, progress), and it keeps a full log it can search when a question comes up that it didn’t predict. A memory is chosen when you write it. A search is chosen when you need it.
2.6 Related research
The most direct evidence on compaction comes from The Complexity Trap, a JetBrains study presented at a NeurIPS 2025 workshop. It also inspired the first tier of my design (section 3).Lindenbauer et al., The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management, DL4Code workshop at NeurIPS 2025 (arXiv 2508.21433). Table 1. They ran a coding agent on SWE-bench Verified with five model setups and three strategies: keep everything, mask old tool outputs with a placeholder, or summarize with an LLM.
Masking cut the cost roughly in half in four of the five setups, and the solve rate mostly stayed within the noise. The LLM summary did no better, and it needs an extra model call. The one clear loss, for both methods, was Gemini Flash in thinking mode. The simplest method is the strongest baseline.
Since then, research has moved along three axes.
When masking helps. It depends on the model. On search agents, masking helped mid-sized models most and did nothing for strong ones, which already ignore stale results. When it cleared something still needed, the agent paid in extra turns.Zhang et al., 2026 (arXiv 2606.00408). See also Min et al., 2026 (arXiv 2608.06503) on compression making agents less stable.
Trimming outputs on arrival. Instead of clearing a whole output once it’s old, keep only the relevant lines from the start. SWE-Pruner cuts 23–54% of tokens on SWE-bench Verified without losing success rate. CoACT cuts 33%, keeping a trim only if the agent’s next action stays the same.
Making summaries safe. A summary can drop something the agent still needs. Four papers attack that risk from different sides:
- Check it before trusting it. Slipstream writes the summary in the background while the agent keeps working, then checks whether it contains what the agent actually used next. Missing pieces get repaired before the switch. Up to +8.8 points.
- Pick a better moment. Summarizing in the middle of a task loses the half-done work. SelfCompact lets the model compact right after finishing a sub-task, when the details are no longer needed. 30–70% cheaper.
- Train for it. Models are trained on full contexts, then suddenly handed summaries. CompactionRL includes compaction during training, so the model learns to write summaries it can work from. +6.6 points.
- Learn from failures. ACON runs the same task with and without compression, finds what the summary lost when only the compressed run failed, and updates its summarizing instructions.
So the simple baseline still holds up, but the frontier has moved from how much to cut to whether the cut was safe.
3. Design: compaction as a view, not a deletion
None of the pieces below is new. What I built is the way they fit together, around one rule: the log keeps everything, and compaction only changes what the model is sent.
- handoff note: goal, done, decisions, rejected approaches, next steps
- your messages, copied word for word
- the agent's notes, copied word for word
- the files it changed
Clear at 50%. Older tool outputs become one line, and the newest stay word for word:
Clearing changes the start of the request, which breaks the provider’s cache (cached input is 10x cheaper, per Manus). So it only runs in batches that save at least 10% of the window.
Notes as it works. A note tool records decisions, rejected approaches and progress when they happen, not reconstructed at the end. Notes live in the log and are copied into every summary.
Summarize at 75% (or 200k tokens). The oldest part becomes one summary, and the request drops to about 40%. The summary has two parts: a handoff note the model writes and updates next time instead of rewriting (an idea from pi), and a part copied by code from the log, so it can’t be paraphrased wrong: the user’s messages, the agent’s notes and the files it changed. Its header tells the model to re-read files for exact details instead of answering from memory.
Recall. The recall tool searches the full log, including everything cleared or summarized. It matches words, also against the call each line came from, so media timeout finds timeout_ms = 1050 in the output of diag.py media.
Put together, this answers the problems from earlier:
| Problem | Where it showed up | What the design does |
|---|---|---|
| The agent forgets what it learned and redoes work | Codex keeps only the user’s messages | Agent messages stay; the handoff note records what’s done and in progress |
| Rejected options get lost | the most common complaint from users | Notes record them when they happen, and every summary includes them |
| Instructions given in chat vanish | Claude Code’s docs, after /compact | The user’s messages are copied word for word into every summary (the newest, up to 5% of the window) |
| Summaries of summaries drift | repeated compaction | The note is updated, never summarized again |
| Memory can’t predict which detail will matter | memory chosen when written | recall searches the full log when the question comes |
| Clearing alone isn’t enough | my measurement: about 2.5x smaller | A summary tier on top, at 75% |
4. Experiment: the delayed-recall test
The papers above check the next action or the final result. None asks what happens when a detail is needed long after it was compressed, which is the first complaint from the introduction. So I built a test for it.
4.1 Setup
The agent runs a diagnostics script on six services. Each report hides five facts (a port, a timeout, three error codes, an owner, a region) among about 90 lines of noise. The values are random for every run, so the model can’t know them in advance. Then the script is deleted: the facts now exist only in the session. The agent answers four questions right away, four more after 3 turns of reading log files, and four more after 6 further turns. Answers are scored by exact match: correct, invented, or “not sure”.
Seven setups: a control with no compaction at all (a 200k window the session never fills), and three compaction setups, each with and without recall. Five runs each, on DeepSeek v4 Flash with a 40k window to force compaction early. The same random seeds are used across setups, so each setup sees the same facts and questions.
4.2 Results
The first chart counts every answer per setup; the second shows how correct answers fall as the facts get older.
4.3 Discussion
- With
recall, no invented answers at all, 0 out of 180. My design matched the uncompacted control, 60 of 60, at the same cost per session ($0.023). - But clearing +
recalldid just as well for 25% less ($0.018). In these sessions, clearing alone kept the context under the window, so the summaries only added cost: an extra request each time, and a cache miss after it. The summary tier is there for longer sessions, where clearing alone runs out (it shrinks a session only about 2.5x). This test is too short to show that. - Clearing alone invented the most, 14 answers out of 60. My guess, untested: the placeholder says the report was read, so the model “remembers” seeing it and fills in the values. After a summary, it more often admitted it didn’t know.
- Aggressive compaction was the most expensive with
recall, $0.032 per session against $0.023: nine summaries per session, plus the searches to recover what they dropped. recallonly works if the model uses it. In one aggressive run, the model never called it and answered “not sure” six times.
The first version of the test also caught a flaw in recall itself. It searched for the exact phrase, and the model asked for things like media timeout_ms, while the report has media in its header and timeout_ms = 1050 forty lines below. In the pilot, it found nothing. I changed it to match words, including the name of the call each line came from, and to rank tool outputs first.
5. What compaction costs
On every turn, the model re-reads the whole context, and you pay for all of it again. So a smaller context should mean a cheaper session. Two things make the saving smaller than it looks:
- The cache. Providers bill the part of the request they’ve already seen at about a tenth of the price. A big context that doesn’t change is cheap to re-read.
- Compaction has its own price. The summary is an extra request, and the request right after it gets no cache discount, because the start of the context has changed.
So is it cheaper to compact often or rarely? I modeled a 150-turn session at Claude Sonnet prices, with three thresholds:
On the bill alone, compacting often wins: 37% cheaper than compacting late. But it writes 18 summaries instead of 4, and each one can lose something.
The delayed-recall test measured what happens next. Once the agent has to search for what the summaries lost, compacting often cost 41% more per session than my design, which summarizes at 75%. That’s why the design clears old outputs first, summarizes late, and relies on recall for the rest.
6. Limitations
- One model. The delayed-recall test ran on DeepSeek v4 Flash only. Stronger models may forget less, or use
recalldifferently. - Synthetic facts and a simulated window. Random service facts and a 40k cap make the test controllable, but real sessions are messier.
- Short sessions. About 14 turns each, so clearing alone was enough and the summary tier never had to prove itself.
- Small samples. Five runs per setup, 60 answers each. Large gaps (0 vs 14 invented answers) are clear; small ones are not.
- The prompt helps
recall. The system prompt tells the model to search instead of answering from memory. Without that line, it may callrecallless often. - The cost curve in section 5 is a model with stated assumptions, not a measurement.
7. Future work
The obvious next test is a much longer session, where clearing alone overflows the window: that’s where the summary tier should earn its cost, or not. My design also uses fixed thresholds: 50%, 75%, 200k. The 2026 papers point to what’s missing. SelfCompact says the timing should follow the work (compact after a sub-task, not in the middle of one), Slipstream says a summary should be checked before the agent relies on it, and ACON says the harness can learn what to keep from its own failures. My agent already logs every session in full, so the data for all three is there. That’s what I want to try next.
If you build agents, I’d like to know how you handle this.
References
- Karpathy, A. Post on X about context engineering. June 2025. x.com
- Liu, N. F. et al. Lost in the Middle: How Language Models Use Long Contexts. TACL, 2023. arXiv 2307.03172
- Modarressi, A. et al. NoLiMa: Long-Context Evaluation Beyond Literal Matching. ICML, 2025. arXiv 2502.05167
- Huang, D. et al. ATLAS: All-round Testing of Long-context Abilities across Scales. 2026. arXiv 2605.28079
- Hong, K., Troynikov, A. and Huber, J. Context Rot: How Increasing Input Tokens Impacts LLM Performance. Chroma, 2025. trychroma.com
- Lindenbauer, T. et al. The Complexity Trap: Simple Observation Masking Is as Efficient as LLM Summarization for Agent Context Management. DL4Code workshop, NeurIPS 2025. arXiv 2508.21433
- Kang, M. et al. ACON: Optimizing Context Compression for Long-horizon LLM Agents. 2025. arXiv 2510.00615
- Min, G. et al. Toward Reliable Context Compression for Long-Horizon Agents: An Empirical Study of Execution Instability. 2026. arXiv 2608.06503
- Zhang, H. et al. Masking Stale Observations Helps Search Agents, Until It Doesn’t: A Regime Map and Its Mechanism. 2026. arXiv 2606.00408
- Wang, Y. et al. SWE-Pruner: Self-Adaptive Context Pruning for Coding Agents. 2026. arXiv 2601.16746
- Chen, H. et al. CoACT: Action-Preserving Observation Compression for Coding Agents. 2026. arXiv 2607.02911
- Chen, Z. et al. Slipstream: Trajectory-Grounded Compaction Validation for Long-Horizon Agents. 2026. arXiv 2605.08580
- Li, T. et al. Self-Compacting Language Model Agents. 2026. arXiv 2606.23525
- Li, Y. et al. CompactionRL: Reinforcement Learning with Context Compaction for Long-Horizon Agents. 2026. arXiv 2607.05378
- Anthropic. Managing context on the Claude Developer Platform. 2025. claude.com
- Anthropic. How Claude remembers your project. Claude Code docs. code.claude.com
- OpenAI. Memories. Codex docs. developers.openai.com
- Google. Gemini CLI v0.40.0: Tiered Memory. 2026. github.com
- Ji, Y. Context Engineering for AI Agents: Lessons from Building Manus. 2025. manus.im