Your Agent's Real Bottleneck Isn't the Prompt. It's the Context.

Ask most teams how they manage what their coding agent sees over a long session, and the honest answer is: they do not, really. Context accumulates — tool outputs, file contents, prior turns — until the window fills or the agent starts making mistakes it would not have made ten turns earlier. That degradation has a name now, context rot, and it is the actual bottleneck on long-horizon agent tasks, not prompt wording.


The framework

Current research and production practice converge on four strategies for managing an agent's context deliberately instead of letting it accumulate by default:

  • Write: persist information outside the active context window instead of keeping it all in-line. The common pattern is a scratchpad the agent writes progress notes to, so it can recover task state without re-reading everything that led up to it.
  • Select: pull in only what is relevant to the current step, using retrieval and relevance scoring instead of dumping an entire file or history into context because it might be useful.
  • Compress: summarize or distill older context instead of carrying it verbatim. Retrieval-augmented approaches like Self-RAG and RAPTOR, and hierarchical memory systems like MemGPT, are examples of this at the systems level.
  • Isolate: hand off narrow, risky, or verbose subtasks to a sub-agent with its own limited context, rather than letting one agent's context balloon with work that is only relevant to a small slice of the overall task.

None of these are exotic. What is new is treating them as a deliberate architecture decision rather than something that happens by accident of how a framework's default agent loop is built.

Why this is showing up in the research now

A January 2026 paper, Meta Context Engineering via Agentic Skill Evolution (Ye et al.), reports that optimizing context assembly directly — rather than hand-tuning prompts — produced an 89.1% average relative improvement over a base model on SWE-bench Verified in an offline setting, against 70.7% for the next-best hand-engineered baseline, and the method kept its lead in an online setting too (74.1% versus 41.1%). The specific numbers matter less than what they demonstrate: context assembly is now measurable as a distinct variable from model choice or prompt wording, and optimizing it moves the needle by a wide enough margin that it is no longer a rounding error next to which model you picked.

A separate line of research, on context rot in long-horizon search and long-context reinforcement learning, is documenting the failure mode from the other direction: agent quality measurably degrades as context grows, even when the underlying model has a large context window and technically has room for everything. A big context window solves the fitting problem, not the attention problem — a model with a million tokens of context available does not necessarily reason as well over token 900,000 as it does over token 9,000.

What this looks like in practice, today

You do not need to reproduce a research paper to apply this. Concrete, shippable patterns for each strategy:

  • Write: give your agent an explicit scratchpad tool, separate from its working context, and have it write short progress notes at natural checkpoints (task started, subtask complete, decision made and why). On resume or recovery, read the scratchpad back in instead of replaying the full transcript.
  • Select: before dumping a file or a tool's full output into context, ask whether the agent needs all of it for the current step or just a slice. A search-and-extract step that pulls the three relevant functions out of a 2,000-line file beats pasting the whole file, both for context budget and for the model's ability to actually attend to what matters.
  • Compress: set a periodic summarization checkpoint — every N turns or every completed subtask, collapse the detailed history into a short structured summary and drop the verbose version. Treat this as a scheduled maintenance step, not something that only happens when you hit a context limit error.
  • Isolate: when a subtask is going to be verbose (reading a large log file, iterating on a flaky test, exploring an unfamiliar part of a codebase), hand it to a sub-agent with a narrow, purpose-built context rather than letting that exploration bloat your main agent's window. Bring back only the sub-agent's conclusion, not its full working transcript.

The practical takeaway

If your team has been treating context management as "whatever the framework's default agent loop does," that is very likely costing you reliability on anything longer than a short, single-step task — and it is a cheaper fix than most alternatives. Before reaching for a bigger context window or a more expensive model to solve a long-horizon task that is degrading in quality, look at whether you are writing, selecting, compressing, and isolating context deliberately, or just letting it accumulate. In the research cited here, the gap between doing this well and doing it by default was larger than the gap between models.

Sources

This story was written with the assistance of an AI writing program.

Comments

Post a Comment

Popular posts from this blog

GitHub's New Copilot Mode Picks a Different Model for Every Step of Your Task

GitHub's Project HydraFusion doesn't pick one model for your coding task — it builds a small execution plan per task and routes across models mid-task. It's a formalization of patterns a lot of teams already hand-roll in production agents, which makes it a useful reference architecture even if you never turn it on. What it actually is On September 4–5, 2026, GitHub shipped Project HydraFusion as a research preview inside Copilot CLI, accessible via an /experimental flag to users on any Copilot plan ( GitHub Blog ). Rather than statically routing a task to one chosen model, HydraFusion builds a runtime execution plan per task and selects from one of three patterns ( MarkTechPost ): Single — one selected model solves the task directly, no orchestration overhead. The default for tasks the router judges simple. Cascade — a cheaper, faster model drafts a solution first; a quality gate evaluates the draft and either accepts it or escalates the task to a stronger (more exp...
Read more

An Attacker's Multi-Agent Framework Stole Thousands of Credentials in Under Six Hours

Image
A financially motivated attacker ran reconnaissance, exploitation, and cleanup with almost no human in the loop — using the same design patterns you'd use to build a helpful agent. That symmetry is the actual story. What Google's threat researchers found Google's Threat Intelligence Group (GTIG) published its Q3 2026 AI Threat Tracker documenting a mass credential-harvesting campaign carried out largely by a multi-agent framework rather than a human operator. According to GTIG's report and coverage from The Hacker News, SiliconANGLE, and Help Net Security, a suspected financially motivated actor gained access to an organization's cloud infrastructure, then deployed an autonomous multi-agent system from inside that environment — which let requests originate from legitimate-looking IP addresses rather than obviously malicious infrastructure. The attacker reportedly built the framework using an AI coding chatbot, a set of instructions, and preconfigured Markdown fil...
Read more

A Prompt Injection Turned Into a Shell: Inside Semantic Kernel's Two RCE CVEs

Image
Two ordinary agent-framework conveniences, a filterable vector search and a file-download tool, turned into remote code execution once an LLM's output was trusted a little too much. Here's what happened in Microsoft Semantic Kernel, and what to check in your own agent code today. Prompt injection usually gets discussed as an output-quality problem: the model says something it shouldn't, or does something a user didn't ask for. CVE-2026-26030 and CVE-2026-25592, two vulnerabilities Microsoft disclosed in its Semantic Kernel framework, are a sharper reminder of what it can become: full remote code execution, demonstrated in Microsoft's own writeup by launching calc.exe on the machine running the agent. Semantic Kernel is Microsoft's open-source framework for building agents and wiring LLMs into tool-calling applications, widely used both inside Microsoft's own ecosystem and in third-party projects. Both vulnerabilities follow the same underlying pattern in ...
Read more