Agents that call tools in a loop work well for short jobs, but when a task stretches to an hour and requires hundreds of tool calls the simple loop breaks in two predictable ways: the context window overflows and the original goal gets lost. The fix is not a bigger model or a larger context window; it is the harness that manages everything except the model itself.
First, the harness decides what never enters the window. Offloading rules move large tool results to disk and replace them with short pointers. When the session context reaches a set percentage of the model’s window, older write and edit calls are truncated to file paths, keeping only a preview. Only after offloading is exhausted does the system fall back to summarization. This keeps the window from filling with raw observations that dilute attention.
Second, when offloading cannot keep up, the harness compacts the conversation. It summarizes the current context and starts a new window with that summary. Effective compaction preserves the intent, unresolved bugs, and next steps while discarding redundant tool outputs. Some implementations store the full transcript on disk so any lost detail can be retrieved later, and they reload a few recently modified files after each compaction to keep the plan fresh.
Third, todo‑state recites the goal on every turn. By maintaining a todo.md file that is rewritten and checked off step by step, the agent pushes the global plan to the end of the context, counteracting the “lost in the middle” drift. The file survives compaction and resets, acting as a mutable artifact of the objective.
Fourth, persistent memory across sessions stores rules, notes, or extracted facts outside the window and reloads them after each compaction or reset. This prevents re‑researching known information, but it adds a small inference cost that should be kept minimal by limiting the size of persistent files.
Together, these four mechanisms—context budgeting and offloading, goal‑aware compaction, todo‑state recitation, and disciplined memory—turn a shallow agent loop into a deep agent that can run for hours without losing its goal or overflowing its context.
#AI #Productivity #LLM #AgentEngineering #ContextManagement #DevTools