A Context Engineering Checklist for Long Agent Runs

A context engineering checklist is a set of true-or-false statements about what enters an agent’s context window on every call: which sources, in what order, under what token budget, and what happens when the budget runs out. This one has 40 lines in ten groups, each tagged with its operation and the chapter that explains it.

I wrote this context engineering checklist for a tech lead whose team already runs an agent and manages its context by folklore. One engineer restarts at a size they have learned to distrust, another lets the harness summarize, a third never compacts. By the end you should have three things to post in the team channel: the ticked list, a budget that adds up, and a filled-in policy block.

How do you use a context engineering checklist on one recorded run?

Take one recorded run of your own system, one hour, and tick only what the evidence shows. A line passes when the place it names (the trace, the prompt assembly code, the memory store or the policy block) shows the statement to be true. “We intend to” and “I can’t tell” both count as no. When a run delegates, tick the list for the parent window, then repeat the first group and the tool-results group for one worker, with that worker’s own ceiling.

Pick a long run: one that reached a late step, used tools and, if your system compacts, was compacted at least once. A line about an event the run never contained gets “not applicable” with the reason written beside it, and “we keep no durable memory, by decision” is a passing reason. Lines that fail become tickets, each with a chapter to read.

Every line carries one of the four operations of context engineering, which are write, select, compress and isolate, or the tag “none”. The tags, the grouping and the pass conditions are my arrangement; the practices come from Chapters 2, 5, 7, 8, 9 and 19 of AI Agents, Engineered unless a line says otherwise.

Eleven of the 40 lines in this context engineering checklist carry no operation. Nine cover ownership, order, the budget and verification, one is the grounding instruction for retrieval and one asks where the memory store came from. A list built only from the four verbs would miss all eleven.

Can you see and count what the model read?

You can if any call’s assembled input prints from a stored trace with a token count per component. Every later group depends on this one, because a line you cannot check against a recorded window is an opinion. Chapter 7 calls the assembled input “a first-class artifact of your system, worth inspecting, versioning, and testing like code”.

The same chapter says where the ceiling comes from: “The only trustworthy way to find the soft limit for your model and your task is to measure it. The spec sheet will not tell you.” The inventory of a single call is a separate job, and the per-call manifest in context engineering vs prompt engineering is the form for it.

What does a budget that adds up look like?

A budget adds up when every component has a cap, the caps sum to less than the ceiling, and the gap divided by the growth per step gives the number of steps before a trim must fire. The first group of the context engineering checklist ends on that arithmetic, and the example below is illustrative: every token figure is a round number I invented, including the window size.

A cap in this table is the size a trim brings a component back to, and the headroom is how far the pile may overshoot the caps between trims. The sum is 2,000 + 500 + 20,000 + 1,500 + 12,000 + 6,000 + 15,000 + 700 + 300 = 58,000 tokens. It is the planner’s sum of my invented caps, and no chapter prints it.

Take an invented window of 200,000 tokens and a 40% ceiling, and the ceiling is 0.40 × 200,000 = 80,000 tokens. Utilization right after a trim is 58,000 / 200,000 = 29%, and the headroom is 80,000 − 58,000 = 22,000.

If each step adds 2,000 untrimmed tokens, the run reaches the ceiling in 22,000 / 2,000 = 11 steps and the hard limit in ⌊(200,000 − 58,000) / 2,000⌋ = 71. The policy reading is the first number: a trim or a compaction fires by the 11th step. The 40% is the book’s illustration of a practitioners’ figure, which Chapter 7 calls “a practitioner’s rule of thumb for particular models and tasks, offered here as an illustration of the practice, never as a constant”. Replace it with the ceiling you measured.

What does the planner show for these numbers?

The planner below opens on the budget above and shows 29% utilization, 22,000 tokens of headroom, about 11 steps to the ceiling and about 71 to the hard limit. It marks history, tool results and retrieved documents for an operation, since each holds 15% or more of the pile.

It models growth with no trim applied, so 11 is the longest interval between trims. The neighboring post on context engineering vs prompt engineering runs the same planner on a single recorded call, to show what share of one window a person wrote; here it holds a policy for a whole run, which is why the seed differs. That post’s table of changes by kind of system gives the reason for a practice, and the lines here give its pass condition.

Why cap the component as well as each result?

A cap on the component catches what a limit on each item misses. Suppose each tool result has a hard limit of 5,000 tokens, one that a single result may never exceed. Five results of 4,000 tokens each pass that guard and together hold 5 × 4,000 = 20,000 tokens, which is 5,000 over the 15,000 cap for tool results. That overshoot comes out of the 22,000 headroom until a trim restores the cap.

One open issue on an open-source personal-assistant agent, filed in July 2026, reports this shape: “multiple medium-sized results that collectively overflow the context are not caught”. The context window budget planner takes your own nine counts on its own page.

Is the window assembled in the right order?

The order is right when stable content comes first, variable content comes last, and the instruction follows any long material. Two rules from the book cover it. Chapter 19 gives the caching rule, “stable content first, variable content last”, and Chapter 2, which is free to read, gives the placement rule: “The last thing the model reads should be the thing you most need it to do.”

The two rules fit together. Standing instructions sit at the top and stay byte-identical, and the one rule that carries the load is restated as a fixed line at the tail. One agent product’s team wrote in 2025 that “even a single-token difference can invalidate the cache from that token onward” (Ji, 2025), and their example was a timestamp at the top of the system prompt.

A trim or a compaction rewrites earlier turns, so each one costs a cache rebuild from the point of the rewrite. That is an argument for batching them at boundaries, and the prompt caching savings calculator prices the difference.

What belongs in the standing instructions and the tool list?

Only what almost every task needs belongs there, because both are read on every pass of every run. Chapter 9 calls the always-loaded core the hot tier and gives its rule in four words: “keep it ruthlessly small”. The same chapter treats that file as procedural memory, which is why its edits get review: “A wrong fact costs you an answer. A wrong rule costs you the agent.”

The question of when a tool should become a skill covers which procedures move out of the standing file and which rules must stay. One line here points at it.

Which tools should a run carry?

A run should carry the tools its task needs and keep that set for the whole run. Chapter 7 names the first half the tool loadout: “expose only the subset of tools relevant to this task”. The test for ambiguity comes from a 2025 engineering post by Anthropic: “If a human engineer can’t definitively say which tool should be used in a given situation, an AI agent can’t be expected to do better.”

The third line below rests on a source outside the book. The same 2025 product team advises: “unless absolutely necessary, avoid dynamically adding or removing tools mid-iteration”. Tool definitions sit near the front of the window, so a change there invalidates the cache behind it, and earlier steps then refer to tools that are no longer defined.

Retrieval or a long window: how do you choose?

Paste the corpus whole when it is bounded, fits under your measured ceiling and needs close reading; retrieve when any one of four conditions holds. Chapter 8 lists them: “the corpus outgrows the desk; it changes faster than you care to re-paste; it is private and must remain so; or the answers need receipts”. Only the first is about size.

The book’s rule compresses the table: “Retrieval for what the model should know; fine-tuning for how it should respond; the long window for close reading of a bounded set.” Its defense of the long window is one sentence: “Retrieval hands the model fragments; the long window hands it the book.”

What does the evidence say about “RAG is dead”?

The published comparisons support both sides of the argument, and neither supports a slogan. Li and colleagues (2024) compared the two on models of that year: “when resourced sufficiently, LC consistently outperforms RAG in terms of average performance. However, RAG’s significantly lower cost remains a distinct advantage.” LC is their abbreviation for long context.

More passages do not rescue either approach. Jin and colleagues (2024) found that for many long-context models “the quality of generated output initially improves first, but then subsequently declines as the number of retrieved passages increases”. Chapter 8 says it in one sentence: “More retrieved context is not better context.”

So the commenter who wrote in January 2026 “i thought rag/embeddings were dead with the large context windows”, and blamed a chatbot for the idea, had been given the answer to one question of four. A window of any size leaves change, privacy and receipts where they were.

Search the agent runs for itself counts as retrieval in this context engineering checklist, and that includes code search. A system with no index answers the first and the last line, answers the fourth if its answers are meant to come from fetched material, and marks the rest “not applicable, because there is no index”.

Who may write to the agent’s memory?

Each kind of memory gets its own writer: the agent may append to its run log, write facts through a filter, and propose changes to procedures that a person reviews. That split is my reading of Chapter 9, which ranks the three kinds by how dangerous a bad write is and puts the rule for untrusted input plainly: “filtering belongs on the write path, before persistence”.

The first two lines apply to every long run, with or without a durable store. Chapter 9: “Working state belongs to the run; durable knowledge belongs to the relationship.” If you keep no durable store, mark the other five “not applicable, by decision”. Chapter 7 backs that position: “no paging memory system should exist before an observed failure asks for it”.

The measurements are of models and systems from 2024. Wu and colleagues (LongMemEval) report “a 30% accuracy drop on memorizing information across sustained interactions” for commercial assistants and long-context models. Chen and colleagues (AgentPoison) demonstrated an attack with “an average attack success rate higher than 80%” at “a poison rate less than 0.1%”; that is a result under attack conditions and says nothing about how often poisoning happens.

A later paper (Dong and colleagues, 2025) injects records “by only interacting with the agent via queries and output observations”. The everyday version is duller. One commenter found in April 2026 that some entries an agent had written to its own memory file were wrong, and that managing those files was “more than I can expect people on my team to manage on their own”.

What happens to tool results once they have been used?

They are cleared, leaving a one-line fact and a reference. Chapter 7 gives the move: “once a result has been acted on, clear the payload, keep the one-line fact that the call happened, and keep the reference so re-fetching is cheap”. This is the group with the best outside measurement.

Lindenbauer and colleagues (2025) compared two ways of shrinking an agent’s history on one coding benchmark, across five model configurations. They found that “a simple environment observation masking strategy halves cost relative to the raw agent while matching, and sometimes slightly exceeding, the solve rate of LLM summarization”. It is one benchmark of coding tasks.

Sources disagree about resolved errors. The public guide 12-Factor Agents advises: “Consider hiding errors and failed calls from context window once they are resolved.” Chapter 7 agrees, while the 2025 product team quoted earlier says to “leave the wrong turns in the context”.

My reading: keep a failure visible while the same action could still be retried, then replace it with a one-line fact. The line below asks only that your team has decided.

When should the history be compacted, and what must survive?

Compaction should fire below your measured ceiling, at a phase boundary, after the plan is in a file. Chapter 9 gives the reason for firing early: near the limit, “the judgment you are relying on to rescue the run is the judgment the overfull window has already degraded”. The book’s free glossary calls compaction “lossy by design, so what the summary drops is a design decision”.

On what survives, Chapter 9 is specific: “What must survive a compaction hardly varies across reports: the decisions made and why, the problems still open, the current plan, and the handful of artifacts under active work.”

Here the evidence runs out. I found no measurement of how often a compaction drops a constraint, and none of quality by trigger point. What exists is a volume of reports, such as this open issue from September 2026: “requirements I stated at the start of a session, and constraints I set mid-session, stop being followed”. Treat any percentage in your policy as a value to measure on your own tasks.

Harness-provided compaction is a category with a trigger somebody chose for you. Two examples, both read on 2026-10-06: one model provider’s API documentation (OpenAI) describes a compaction item that is “opaque and not intended to be human-interpretable”, and one agent framework’s documentation (Google’s Agent Development Kit) summarizes older history “including instructions, inputs, and model responses”. A summary nobody can read cannot be reviewed, and a summary that covers instructions can paraphrase a rule. That is one of the mechanical reasons why AI agents ignore instructions late in a run.

When does work go to a separate window?

Work goes to a subagent when it is bulky and separable, or when the content is untrusted; it stays in the main thread when the pieces share context that keeps changing. Chapter 9 calls the pattern quarantine and prices it: total token spend multiplies, and tightly coupled work split across sealed windows comes back as fragments.

The rule for the boundary is the chapter’s quarantine rule, printed as a display line: “Nothing enters an isolated context except what its brief deliberately admits, and nothing comes back out except a distilled result sized for the desk that will read it—carried as evidence, never as orders.” A system with no delegation marks the first three lines of this group “not applicable”; the fourth still applies to any agent that reads content it does not control.

How do you know a trim did no harm?

You know when a test would have failed had the trim dropped something a later step needed. Chapter 9 explains why reading the output will miss it: “a compression that quietly hurts task success looks identical, from the inside, to one that saved the run”. Replay and step-wise grading are practices this cluster of posts adds to the book’s advice to measure.

The last line borrows the clean-desk test, which belongs to the post on context rot and its four fixes; the five-minute explainer shows the same ground in motion.

What goes in the context policy block?

The policy block holds the choices the context engineering checklist asks about: the ceiling, the caps, the order, the trims, the compaction trigger and survival list, the retrieval decision, the memory writers, the isolation rule and the checks. It is my synthesis of Chapters 7, 8 and 9. Fill it in for one agent and commit it beside the assembly code.

CONTEXT POLICY: [agent or feature] · owner: [name] · reviewed: [date]

Budget
  Utilization ceiling: [ ]% of the window, measured on [task set] on [date]
  Caps (tokens): instructions [ ] · request [ ] · history [ ] · memory [ ] ·
    retrieved passages [ ] · tool definitions [ ] · tool results [ ] ·
    examples [ ] · output shape [ ]
  Sum of caps: [ ] (must be under the ceiling) · growth per step: [ ]
  Steps before a trim or compaction must fire: (ceiling - sum) / growth = [ ]
  Owner per component: [ ]

Order
  Stable first: [what never changes within a run]
  Variable last: [what changes per call]
  Load-bearing rule restated at the tail: [the rule]

Standing instructions and tools
  Always loaded: [file] · loaded on demand: [library]
  Edits reviewed by: [ ] · tool loadout per task type: [name: tools, tokens]

Tool results
  Hard cap per result: [ ] tokens · past the cap: [file + slice + pointer]
  Cleared after: [ ] steps | never, for runs under [ ] steps · leaves: fact + reference
  Resolved errors: [hidden after ... | kept because ...]

History
  Compaction fires at: [phase boundary], and no later than [ ]% · measured on: [task set]
  Survives word for word: constraints, standing rules (from [file]) ·
    put back by: [harness reinserts | agent rereads first]
  Survives in the summary: decisions and why, open problems, current plan,
    artifacts in progress
  Start fresh from the notes when: [signs]

Retrieval
  Decision: [long window | retrieval | both | not applicable]
  Outgrows the window? [ ] · changes? [ ] · private? [ ] · needs receipts? [ ]
  Index: [yes | none, agent search only] · grounding instruction for: [which questions]
  Passages per call after reranking: [ ] · filters: [ ] · keyword search: [ ]
  Agent-directed search: hop cap [ ] · spend cap [ ]

Memory
  Run state lives in: [file] · durable facts live in: [store | none, by decision]
  Who may write: run log [agent] · facts [agent, filtered] · procedures [human review]
  Expiry: [ ] · contradiction rule: [ ] · scoped per: [user | project]

Isolation
  Goes to a separate window when: [bulky and separable | not applicable]
  Untrusted content is read by: [worker, minimal tools | none is read, because ...]
  Brief carries: objective, result shape, tools and sources, boundaries
  Return capped at: [ ] tokens, treated as evidence

Checks
  Replay set: [ ] recorded windows · graded at steps: [early, middle, late]
  Checklist last run on trace: [id] · lines failed: [ ] · tickets: [ ]

What can the checklist not tell you?

A context engineering checklist cannot tell you whether the agent is good at its task, and it cannot supply a number. Every line can pass on a system that still fails, because the list checks that decisions were made and are visible. Whether they were the right decisions is a question for your evals.

Most lines rest on the book’s reasoning and on practice. Measurement exists for input length, position, passage count, tool count and one comparison of masking against summarizing. I found none for compaction triggers, constraint survival or memory expiry rules. The six studies cited used models from 2024 and 2025, I read only their abstracts, and thresholds move with each generation.

Some systems should mark most of the list “not applicable”. A single call has no history to compact and no results to trim, so the first two groups, the standing instructions and, if it retrieves, the retrieval group are the whole audit. A run of a dozen steps that stays far under its ceiling can skip compaction and the first three isolation lines.

Over-curation is the opposite failure, and Chapter 7 names its price: “The trade is always recall against focus”. My harvest of reader questions was mostly from coding agents, so support and research agents may weigh the groups differently.

One recorded run, one hour

The use of a context engineering checklist is to turn “how we manage context” from habit into a page somebody can review. Chapter 9 closes its account of long runs with the line I would put at the top of that page: “a healthy long run is a chain of short runs joined by good notes”. Run the list on one trace this week, fill in the block, and file a ticket for each line that failed.

The order rule is in Chapter 2, free to read, and the glossary is free too. Chapter 7, “Managing the Context Window”, Chapter 8, “Retrieval and Knowledge” and Chapter 9, “Memory: Working State Across Long Runs” are in the full book, as are Chapter 5 and Chapter 19, where the response cap and the caching rule live. The book is written in pseudocode throughout, so nothing in it depends on one harness. The context engineering guide collects the neighboring posts and tools, or you can see the formats.