What is context engineering?
Context engineering is deciding what goes into a language model's context window on every call: the instructions, the tool list, the conversation so far, tool results, retrieved documents, notes and memory. Then keeping that set small and useful as an agent works through a long task.
It matters because the model knows nothing except its training and the text you send right now. An agent calls the model again and again, and each call needs a fresh prompt built from everything that has happened. Someone, or some code, has to decide what makes the cut each time.
Think of the context window as a whiteboard in a meeting room. The model is a brilliant consultant who walks in, reads only the whiteboard, says one thing, and leaves. Then the next consultant walks in. Prompt engineering is writing the brief at the top of the board well. Context engineering is managing the whole board between visits: what to add, what to rub out, what to file in a folder and leave a pointer to, and what to rewrite as a short summary.
Anthropic's engineering team put the goal in one line on 29 September 2025: good context engineering means "finding the smallest possible set of high-signal tokens that maximize the likelihood of some desired outcome." Andrej Karpathy, who helped popularise the term in June 2025, called it "the delicate art and science of filling the context window with just the right information for the next step."
Context engineering vs prompt engineering
Prompt engineering is part of context engineering. LangChain's Harrison Chase says it plainly: "prompt engineering is a subset of context engineering." The difference is scope and timing.
| Prompt engineering | Context engineering | |
|---|---|---|
| What you work on | The instructions: system prompt, task wording, examples | Everything the model sees: instructions plus tools, history, tool results, retrieved data, memory |
| When | Written once, then tested and revised | Decided again before every model call, by code |
| Typical question | "How do I phrase this so the model follows it?" | "After 40 tool calls, what should still be in the prompt?" |
| Typical fixes | Clearer instructions, better examples, output format | Clearing old tool output, summarising, notes, retrieval, sub-agents, cache-friendly layout |
| Main failure | The model misunderstands the task | The model has the wrong information: missing, buried, outdated or contradicted |
A quick example. A support bot answers a refund question wrongly. If the policy text was in the prompt and the bot misread it, that is a prompt problem: rewrite the instruction. If the policy was never retrieved, or was retrieved next to an older version, or was pushed out of the window by 30 earlier messages, that is a context problem, and no amount of rewording fixes it. Chase's view is that most agent failures are the second kind: "Most of the time when an agent is not performing reliably the underlying cause is that the appropriate context, instructions and tools have not been communicated to the model."
Why more context is not better: context rot
Modern models accept hundreds of thousands of tokens, so why not send everything? Three reasons.
- Accuracy drops as the prompt grows. Chroma tested 18 models in July 2025, holding the task fixed and changing only the input length, and found that "their performance grows increasingly unreliable as input length grows." On a long-memory benchmark, every model did significantly better with a focused prompt of about 300 tokens than with the full prompt of about 113,000 tokens. Anthropic calls this context rot and describes it as a gradient, not a cliff: models stay capable but lose precision.
- The middle gets less attention. The 2023 paper "Lost in the Middle" found that performance "is often highest when relevant information occurs at the beginning or end of the input context, and significantly degrades" when the model has to use information from the middle of a long context.
- Cost and speed scale with input. An agent re-sends its whole context on every call and writes only a short reply. Manus, a general-purpose agent, reports an average input-to-output ratio of "around 100:1" and about 50 tool calls per task. Input tokens are the bill.
LangChain names four ways a long context goes wrong: poisoning (a hallucination gets into the context and is treated as fact), distraction (so much context that the model leans on it instead of its training), confusion (irrelevant content steers the answer) and clash (parts of the context disagree). Each one is a reason to curate rather than accumulate.
Context engineering techniques
LangChain groups the techniques into four verbs: write context outside the window, select what to pull in, compress what is there, and isolate work into separate contexts. Here is what each looks like in an agent.
| Technique | What it does | Use it when | Watch out for |
|---|---|---|---|
| Just-in-time retrieval (select) | Keep file paths, URLs or query names in context and load the content with a tool only when needed | Large codebases, document sets, databases | Slower than having it ready; the agent must know where to look |
| Tool-result clearing (compress) | Replace old, bulky tool output with a short placeholder; keep the calls and the agent's own reasoning | Long tool-heavy runs; the safest first step | Facts the agent read but never restated disappear |
| Compaction (compress) | Near the limit, summarise the conversation and continue from the summary plus recent messages | Long back-and-forth tasks | The summary is a one-time bet on what will matter later |
| Structured notes (write) | The agent saves findings, plans or a to-do list to a file outside the window and reads it back | Multi-step work with milestones | Extra calls; the model must actually write the notes |
| Sub-agents (isolate) | Hand a sub-task to a fresh agent with its own window; get back a short summary | Research and exploration that can run in parallel | Many more tokens overall; information lost in the hand-off |
| Cache-friendly layout | Keep the start of the prompt identical between calls and only append | Always | Timestamps, reordered JSON keys or edited history at the top break the cache |
| Lean tool sets | Fewer, clearly separated tools, or retrieving tool descriptions on demand | Agents with many tools or MCP servers | Overlapping tools make the model guess |
Three lessons from Manus's July 2025 write-up are worth knowing by name, because they go against first instincts:
- Mask tools, don't remove them. Tool definitions usually sit at the start of the prompt, so adding or removing tools mid-task breaks the cache for everything after them. Manus keeps the list fixed and instead restricts which tools the model may pick at each step.
- Keep the wrong turns in. When the model sees its failed action and the error, it is less likely to repeat it. Cleaning errors out of the history hides the evidence.
- Recite the plan. Manus keeps rewriting a to-do list file as it works, which puts the goal at the end of the context, where the model attends best, instead of buried at the start.
Anthropic's own rule of thumb for choosing: compaction for long conversations, note-taking for iterative work with clear milestones, and multi-agent setups for research where parallel exploration pays off. Sub-agents typically return "a condensed, distilled summary of its work (often 1,000-2,000 tokens)".
We tested six context strategies on one agent run
Advice on context engineering is easy to agree with and hard to picture. So we took one agent run and built, call by call, the exact prompt that six common strategies would send to the model.
The run: a coding agent is debugging 500 errors on ShopKart's checkout API (invented). It makes 30 tool calls: it reads logs, searches the code, opens files, runs SQL queries and checks config. Most tool output is routine noise, as in real runs. Hidden in it are five facts the final answer needs:
| Fact | Found at step | What it says | Agent restated it? |
|---|---|---|---|
| F1 | 3 | Errors began right after build 7.4.2 was deployed | Yes |
| F2 | 7 | Commit a41f9c2 moved coupon logic into a shared function | No ("Useful. Next, the cart service.") |
| F3 | 12 | checkout.controller.ts line 141 applies the discount a second time | Yes |
| F4 | 18 | 1,284 of 1,301 failures are coupon orders over ₹5,000 | No ("Useful. Checking config next.") |
| F5 | 24 | The DOUBLE_DISCOUNT_GUARD flag is off in production | Yes |
That mix is deliberate and realistic: agents often read something important and move on without restating it. We wrote the run before testing any strategy and did not change it afterwards. The model is called 31 times: once before each tool call and once for the final answer. For each call, each strategy builds its prompt:
- keep_all: the system prompt, the tool list and every message so far. The default in most agent loops.
- timestamp: the same as keep_all, but the system prompt starts with the current time to the second. A common habit ("so the agent knows the date").
- window: only the last 10 messages (5 tool calls and their results).
- clearing: once the full prompt passes 32,000 characters, every tool result except the last 3 is replaced by a one-line placeholder. The agent's own messages and tool calls stay. This is what tool-result clearing in the Claude API does, scaled down.
- compaction: once the prompt passes 32,000 characters, everything except the last 4 messages is replaced by a summary. Our stand-in summary is every line the agent wrote, word for word, and none of the tool output.
- notes: clearing, plus the agent saves each finding with a
write_notetool right after it reads it (5 extra calls). The note text is part of the tool call, so clearing keeps it.
For every prompt we measure its size, how much of it is an exact copy of the start of the previous call's prompt (the part a prompt cache can reuse), the input cost at Claude Sonnet 5.5 prices on 21 November 2026 ($2 per million input tokens; with caching, $2.50 to write to the cache and $0.10 to read from it; about 4 characters per token), and which of the five facts are still in the final prompt.
The whole test is two scripts with no libraries: make_trajectory.py writes the run, context.py holds the six strategies and the measurements. Each strategy is a function that takes the history so far and returns the prompt text, which is all context engineering is, in code.
context.py
"""Six ways to build an agent's prompt from the same 30-step run, measured.
No language model is called. For every model call in the run we build the exact prompt text each
strategy would send, then measure: its size, how much of it is an exact prefix of the previous
call's prompt (the part a prompt cache can reuse), an estimated input cost, and which of the five
facts the final answer needs are still in the final prompt.
"""
import json
T = json.load(open("data/trajectory.json"))
FACTS = T["facts"]
TRIGGER = 32_000 # chars (~8,000 tokens): when clearing or compaction kicks in
KEEP = 3 # tool results kept in full when clearing
WINDOW = 10 # messages kept by the sliding window
MIN_CACHE = 2_048 # chars (~512 tokens): shorter prefixes are not cached
# USD per million input tokens, Claude Sonnet 5.5 list prices on 21 Nov 2026 (see sources):
# normal input, 5-minute cache write, cache read. Tokens ~ chars / 4.
BASE, WRITE, READ = 2.00, 2.50, 0.10
def head(stamp=None):
tools = "\n".join(f"- {t['name']}: {t['description']}" for t in T["tools"])
top = f"Current time: {stamp}\n" if stamp else ""
return f"{top}[system]\n{T['system']}\n\n[tools]\n{tools}\n"
def render(m):
if m["role"] == "assistant":
return f"\n[assistant]\n{m['text']}\n<call {m['tool']} {json.dumps(m['args'])}>\n"
if m["role"] == "summary":
return f"\n[summary of earlier work]\n{m['text']}\n"
return f"\n[tool result: {m['tool']}]\n{m['text']}\n"
def cleared(m):
return {**m, "text": f"[result cleared to save space: {len(m['text']):,} chars]"}
def calls(msgs):
"""Indices where the model is called: before each assistant message, plus the final answer."""
return [i for i, m in enumerate(msgs) if m["role"] == "assistant"] + [len(msgs)]
def run(name, msgs, build):
prompts = [build(msgs[:i], k) for k, i in enumerate(calls(msgs))]
return name, prompts
def keep_all(hist, k):
return head() + "".join(map(render, hist))
def timestamp(hist, k):
# Same content as keep_all, but the system prompt starts with the time to the second.
s = 7 + 37 * k
return head(f"2026-11-18 15:{(s // 60) % 60:02d}:{s % 60:02d}") + "".join(map(render, hist))
def window(hist, k):
return head() + "".join(map(render, hist[-WINDOW:]))
def clearing(hist, k):
# Like server-side tool-result clearing: once the full prompt passes TRIGGER, every tool
# result except the last KEEP is replaced by a placeholder. Tool calls and the agent's own
# words stay. Decided again on every call from the full history.
full = head() + "".join(map(render, hist))
if len(full) <= TRIGGER:
return full
tool_idx = [i for i, m in enumerate(hist) if m["role"] == "tool"]
drop = set(tool_idx[:-KEEP])
return head() + "".join(render(cleared(m) if i in drop else m) for i, m in enumerate(hist))
class Compaction:
# Like client-side compaction: when the prompt passes TRIGGER, everything except the last 4
# messages is replaced by one summary. Our stand-in summary is the best case: every line the
# agent wrote, verbatim, and none of the tool output. It can trigger again later.
def __init__(self):
self.summary, self.cut = None, 0
def __call__(self, hist, k):
def build():
body = hist[self.cut:]
pre = [{"role": "summary", "text": self.summary}] if self.summary else []
return head() + "".join(map(render, pre + body))
p = build()
if len(p) > TRIGGER and len(hist) - self.cut > 4:
new_cut = len(hist) - 4
lines = [self.summary] if self.summary else []
lines += [f"step {m['step']}: {m['text']} ({m['tool']})" for m in hist[self.cut:new_cut] if m["role"] == "assistant"]
self.summary, self.cut = "\n".join(lines), new_cut
p = build()
return p
def metrics(prompts):
total = cached = written = 0
prev = ""
for p in prompts:
n = 0
lim = min(len(p), len(prev))
while n < lim and p[n] == prev[n]:
n += 1
n = n if n >= MIN_CACHE else 0
total += len(p)
cached += n
written += len(p) - n
prev = p
cost = (cached * READ + written * WRITE) / 4 / 1e6
cost_nocache = total * BASE / 4 / 1e6
final = prompts[-1]
return {
"calls": len(prompts),
"peak": max(map(len, prompts)),
"final": len(final),
"total": total,
"cached_share": round(cached / total, 3),
"cost": round(cost, 4),
"cost_nocache": round(cost_nocache, 4),
"facts": [f for f, v in FACTS.items() if v in final],
"sizes": [len(p) for p in prompts],
}
if __name__ == "__main__":
runs = [
run("keep_all", T["plain"], keep_all),
run("timestamp", T["plain"], timestamp),
run("window", T["plain"], window),
run("clearing", T["plain"], clearing),
run("compaction", T["plain"], Compaction()),
run("notes", T["noted"], clearing),
]
res = {name: metrics(p) for name, p in runs}
res["_final_prompts"] = {name: p[-1] for name, p in runs}
json.dump(res, open("results.json", "w"), indent=1)
print(f"{'strategy':<11} {'calls':>5} {'peak':>7} {'final':>7} {'total sent':>11} {'cached':>7} {'cost $':>8} {'no-cache $':>10} facts in final prompt")
for name, _ in runs:
r = res[name]
print(f"{name:<11} {r['calls']:>5} {r['peak']:>7,} {r['final']:>7,} {r['total']:>11,} {r['cached_share']:>7.1%} "
f"{r['cost']:>8.4f} {r['cost_nocache']:>10.4f} {len(r['facts'])}/5 {' '.join(r['facts'])}")
make_trajectory.py: the 30-step run and the five planted facts
make_trajectory.py
"""Builds an invented 30-step agent run: a coding agent debugging ShopKart checkout errors.
Every step is one assistant message (with a tool call) and one tool result. Five facts the
final answer needs are planted in tool results at steps 3, 7, 12, 18 and 24. The agent's own
message repeats three of them in its own words; two it only acknowledges ("Useful. Next...").
That mix was written before any strategy was run. Output: data/trajectory.json (deterministic).
"""
import json, random
rng = random.Random(42)
SYSTEM = (
"You are a debugging agent for the ShopKart e-commerce backend. Use the tools to find the root cause "
"of production errors. Read logs, code and data before you conclude. When you are done, reply with the "
"root cause, the evidence and a fix. Never change production data."
)
TOOLS = [
{"name": "read_file", "description": "Read a file from the repository. Args: path, start_line, end_line."},
{"name": "search_code", "description": "Search the repository for a string. Args: query. Returns matching lines."},
{"name": "read_logs", "description": "Read application logs. Args: service, from_time, to_time, level."},
{"name": "run_sql", "description": "Run a read-only SQL query on the analytics replica. Args: query."},
{"name": "git_log", "description": "Show recent commits. Args: path, limit."},
{"name": "write_note", "description": "Append a line to NOTES.md, a file kept outside the conversation. Args: text."},
]
FACTS = {
"F1": "build 7.4.2",
"F2": "a41f9c2",
"F3": "checkout.controller.ts line 141",
"F4": "1,284 of 1,301",
"F5": "DOUBLE_DISCOUNT_GUARD",
}
FACT_LINES = {
3: "ERROR 2026-11-18T14:02:11+05:30 checkout-api POST /v2/checkout 500 first occurrence, deployed build 7.4.2 at 14:00:07",
7: "a41f9c2 2026-11-18 refactor: move coupon logic into applyDiscounts() (shared by cart and checkout)",
12: "141: order.total = applyDiscounts(order, user.coupons); // checkout.controller.ts line 141, cart already applied this",
18: "failed_orders_with_coupon_over_5000 | 1,284 of 1,301 failed checkouts since 14:00 (98.7%)",
24: "DOUBLE_DISCOUNT_GUARD=false # added in 7.1.0, blocks a second discount pass; true in staging, false in prod",
}
# What the agent says after each step. Steps 3, 12, 24 repeat the fact in the agent's own words;
# steps 7 and 18 only acknowledge it. Every other step is routine.
PLAN = [
("read_logs", {"service": "checkout-api", "level": "ERROR", "from_time": "13:30"}, "Checkout is returning 500s. Start with the error logs."),
("read_logs", {"service": "checkout-api", "level": "WARN", "from_time": "13:30"}, "Warnings around the same window, to see what came before the errors."),
("read_logs", {"service": "deployer", "from_time": "13:55"}, "Errors began right after build 7.4.2 went out at 14:00. Checking what that deploy changed."),
("search_code", {"query": "NegativeTotalError"}, "The 500s throw NegativeTotalError. Find where it is raised."),
("read_file", {"path": "src/orders/total.ts", "start_line": 1, "end_line": 80}, "Reading the total calculation."),
("read_file", {"path": "src/orders/total.ts", "start_line": 80, "end_line": 160}, "Continuing the same file."),
("git_log", {"path": "src/orders", "limit": 20}, "Useful. Next, the cart service."),
("read_file", {"path": "src/cart/cart.service.ts", "start_line": 1, "end_line": 120}, "Reading how the cart builds an order."),
("search_code", {"query": "applyDiscounts("}, "Looking for every caller of the discount function."),
("read_file", {"path": "src/cart/cart.service.ts", "start_line": 60, "end_line": 110}, "Cart applies discounts at line 88."),
("read_file", {"path": "src/checkout/checkout.controller.ts", "start_line": 1, "end_line": 100}, "Now the checkout controller."),
("read_file", {"path": "src/checkout/checkout.controller.ts", "start_line": 100, "end_line": 200}, "Found a second call: checkout.controller.ts line 141 applies discounts again after the cart already did."),
("read_file", {"path": "src/coupons/coupon.model.ts", "start_line": 1, "end_line": 90}, "Checking how coupons are stored."),
("run_sql", {"query": "select status, count(*) from orders where created_at > '14:00' group by 1"}, "How many checkouts are failing overall."),
("run_sql", {"query": "select payment_method, count(*) from failed_orders group by 1"}, "Is it tied to a payment method?"),
("read_logs", {"service": "payments", "level": "ERROR", "from_time": "13:30"}, "Payments service errors, to rule it out."),
("run_sql", {"query": "select coupon_id is not null, count(*) from failed_orders group by 1"}, "Do failing orders have coupons?"),
("run_sql", {"query": "select bucket, count(*) from failed_orders_by_total group by 1"}, "Useful. Checking config next."),
("read_file", {"path": "config/production.env", "start_line": 1, "end_line": 60}, "Reading production config."),
("read_file", {"path": "config/staging.env", "start_line": 1, "end_line": 60}, "And staging, to compare."),
("search_code", {"query": "featureFlags."}, "Which feature flags the checkout path reads."),
("read_file", {"path": "src/flags/flags.ts", "start_line": 1, "end_line": 120}, "Reading the flag definitions."),
("read_logs", {"service": "checkout-api", "level": "INFO", "from_time": "13:58"}, "Info logs across the deploy."),
("read_file", {"path": "config/flags.production.yaml", "start_line": 1, "end_line": 80}, "DOUBLE_DISCOUNT_GUARD is off in production and on in staging, which is why staging never failed."),
("read_file", {"path": "tests/checkout.spec.ts", "start_line": 1, "end_line": 120}, "Checking whether tests cover a coupon at checkout."),
("search_code", {"query": "roundPaise("}, "Ruling out a rounding bug in the paise helper."),
("read_logs", {"service": "checkout-api", "level": "ERROR", "from_time": "15:00"}, "Are errors still happening now?"),
("run_sql", {"query": "select count(*) from orders where total < 0"}, "Confirming no negative totals were saved."),
("git_log", {"path": "config", "limit": 10}, "When the flag file last changed."),
("read_file", {"path": "docs/runbooks/rollback.md", "start_line": 1, "end_line": 60}, "Reading the rollback runbook before proposing a fix."),
]
# Routine evidence that a real run would show (not one of the five facts).
EXTRA = {
1: ["ERROR 2026-11-18T14:03:40+05:30 checkout-api POST /v2/checkout 500 NegativeTotalError: order total -412.50 after discounts",
"ERROR 2026-11-18T14:05:02+05:30 checkout-api POST /v2/checkout 500 NegativeTotalError: order total -1,180.00 after discounts"],
9: ["src/cart/cart.service.ts:88: order.total = applyDiscounts(order, user.coupons);",
"src/checkout/checkout.controller.ts:141: order.total = applyDiscounts(order, user.coupons);"],
}
NOTE_AT = {3: "Errors start 14:02, right after build 7.4.2 deployed 14:00.",
7: "Commit a41f9c2 moved coupon logic into applyDiscounts(), shared by cart and checkout.",
12: "Discount applied twice: cart.service.ts line 88 and checkout.controller.ts line 141.",
18: "Only coupon orders over Rs 5,000 fail: 1,284 of 1,301 failures.",
24: "DOUBLE_DISCOUNT_GUARD is false in prod, true in staging."}
SERVICES = ["checkout-api", "cart-svc", "payments", "inventory", "pricing", "auth"]
PATHS = ["/v2/checkout", "/v2/cart", "/v2/cart/items", "/v1/coupons/validate", "/v2/orders", "/health"]
def filler(tool, args, n):
"""Plausible but irrelevant output lines for a tool call."""
out = []
for _ in range(n):
if tool == "read_logs":
out.append(f"{args.get('level') or rng.choice(['INFO', 'WARN', 'DEBUG'])} 2026-11-18T{rng.randint(13, 15)}:{rng.randint(0, 59):02d}:{rng.randint(0, 59):02d}+05:30 "
f"{rng.choice(SERVICES)} {rng.choice(['GET', 'POST'])} {rng.choice(PATHS)} {rng.choice([200, 200, 200, 201, 304, 429])} "
f"{rng.randint(4, 900)}ms req={rng.getrandbits(48):012x}")
elif tool == "read_file":
ln = rng.randint(1, 400)
out.append(f"{ln}: {rng.choice(['const', 'let', 'return', 'if (', 'await', 'export function', '//'])} "
f"{rng.choice(['item', 'cart', 'order', 'user', 'price', 'qty', 'tax', 'shipping'])}"
f"{rng.choice(['.id', '.total', 'Items', 'Price', '.lines', ''])} {rng.choice(['= ', '+= ', '=== ', '&& ', ''])}"
f"{rng.choice(['null', '0', 'true', 'calc(item)', 'items.length', 'res.json(order)'])};")
elif tool == "search_code":
out.append(f"src/{rng.choice(['cart', 'orders', 'checkout', 'pricing', 'flags'])}/{rng.choice(['index', 'utils', 'types', 'service', 'helpers'])}.ts:"
f"{rng.randint(1, 300)}: {rng.choice(['import', 'export', 'const', 'return'])} {rng.choice(['calcTotal', 'roundPaise', 'lineItems', 'taxFor', 'loadFlags'])}(...)")
elif tool == "run_sql":
out.append(f"{rng.choice(['upi', 'card', 'netbanking', 'wallet', 'cod', 'paid', 'failed', 'pending'])} | {rng.randint(1, 9000):,}")
elif tool == "git_log":
out.append(f"{rng.getrandbits(28):07x} 2026-11-{rng.randint(1, 17):02d} {rng.choice(['fix', 'chore', 'feat', 'test'])}: "
f"{rng.choice(['bump deps', 'tidy logging', 'add retry to payments client', 'rename cart fields', 'lint', 'update snapshots'])}")
return out
def build(with_notes):
msgs, step = [], 0
for i, (tool, args, say) in enumerate(PLAN, start=1):
n = rng.randint(14, 40)
lines = filler(tool, args, n)
if i in FACT_LINES:
lines.insert(rng.randint(3, n - 3), FACT_LINES[i])
for x in EXTRA.get(i, []):
lines.insert(rng.randint(3, n - 3), x)
msgs.append({"role": "assistant", "step": i, "text": say, "tool": tool, "args": args})
msgs.append({"role": "tool", "step": i, "tool": tool, "text": "\n".join(lines)})
if with_notes and i in NOTE_AT:
msgs.append({"role": "assistant", "step": i, "text": "Saving that to notes.", "tool": "write_note", "args": {"text": NOTE_AT[i]}})
msgs.append({"role": "tool", "step": i, "tool": "write_note", "text": "ok"})
return msgs
if __name__ == "__main__":
plain = build(False)
rng.seed(42) # same tool outputs for the note-taking run
noted = build(True)
for v in FACTS.values(): # each marker sits in exactly one tool result per run
assert sum(v in m["text"] for m in plain if m["role"] == "tool") == 1, v
json.dump({"system": SYSTEM, "tools": TOOLS, "facts": FACTS, "plain": plain, "noted": noted},
open("data/trajectory.json", "w"), indent=1)
tool_chars = sum(len(m["text"]) for m in plain if m["role"] == "tool")
print(f"steps {len(PLAN)}, messages {len(plain)} (with notes {len(noted)}), tool output {tool_chars:,} chars")
Run it like this. The output below is exactly what we got on 21 November 2026, and it is the same on every run:
Run it (shell)
mkdir -p data && python3 make_trajectory.py && python3 context.py
Output
steps 30, messages 60 (with notes 70), tool output 38,882 chars
strategy calls peak final total sent cached cost $ no-cache $ facts in final prompt
keep_all 31 44,533 44,533 761,081 94.1% 0.0462 0.3805 5/5 F1 F2 F3 F4 F5
timestamp 31 44,567 44,567 762,135 0.0% 0.4763 0.3811 5/5 F1 F2 F3 F4 F5
window 31 12,711 7,978 228,944 13.6% 0.1244 0.1145 0/5
clearing 31 31,842 9,638 499,299 84.8% 0.0580 0.2496 3/5 F1 F3 F5
compaction 31 31,842 16,399 507,875 90.4% 0.0421 0.2539 3/5 F1 F3 F5
notes 36 30,873 10,650 574,515 86.1% 0.0624 0.2873 5/5 F1 F2 F3 F4 F5
What the run shows
| Strategy | Model calls | Final prompt (characters) | Sent over the run (characters) | Reused from cache | Input cost | Facts in final prompt |
|---|---|---|---|---|---|---|
| keep_all | 31 | 44,533 | 761,081 | 94.1% | $0.0462 | 5 of 5 |
| timestamp | 31 | 44,567 | 762,135 | 0.0% | $0.4763 | 5 of 5 |
| window | 31 | 7,978 | 228,944 | 13.6% | $0.1244 | 0 of 5 |
| clearing | 31 | 9,638 | 499,299 | 84.8% | $0.0580 | 3 of 5 |
| compaction | 31 | 16,399 | 507,875 | 90.4% | $0.0421 | 3 of 5 |
| notes | 36 | 10,650 | 574,515 | 86.1% | $0.0624 | 5 of 5 |
Costs are for one run of a small agent; multiply by 1,000 runs a day to see why teams care. "Final prompt" is what the model reads when it writes its answer, which is where missing facts hurt.
1. One line at the top made the same run 10 times more expensive
timestamp and keep_all send the same content, 34 characters apart. But a prompt cache only reuses an exact prefix: the cached part ends at the first character that differs from last time. A clock at the very top changes on every call, so nothing after it is ever reused. Cache reuse fell from 94.1% to 0.0% and the input cost rose from $0.0462 to $0.4763, 10.3 times more. It is even worse than not caching at all ($0.3811), because writing to the cache costs 1.25 times a normal input token. If the agent needs the date, put it at the end of the prompt, or round it to the day.
2. Keeping everything works, and the prompt keeps growing
keep_all had all five facts and, with caching, was one of the cheapest. That is why it is the default. But its prompt grew with every tool call, to 44,533 characters (about 11,000 tokens) at the end, and the five lines that matter are 537 characters of it: about 1% of what the model has to read. Caching makes a long prompt cheap; it does not make it short. This run has 30 tool calls; real coding agents make hundreds, and that is where the context window fills up and long-context accuracy drops (see context rot).
3. A sliding window forgot the whole investigation
The window kept the last 5 tool calls, steps 26 to 30: a rounding check, an error log, a SQL count, a git log and a runbook. None of the five facts was in the final prompt. It was also expensive for its size, because the window's start moves on every call, so only 13.6% could come from the cache. Trimming to recent messages is fine for a chatbot; for an agent doing a long task it throws away the evidence.
4. Clearing and compaction keep what the agent said, not what it read
Both kicked in at call 23. Clearing cut the final prompt to 9,638 characters, 78% smaller than keep_all, and still reused 85% from the cache. Compaction cut the prompt from 31,842 to 4,325 characters in one step and was the cheapest strategy ($0.0421). Both lost the same two facts: F2 (the commit) and F4 (the 1,284 of 1,301 failures), the two the agent read but never wrote in its own words. The three facts it had restated survived, because clearing never touches the agent's messages and our summary copied them.
Our summary is a stand-in. A real model writing the summary reads the tool output too, so it might keep F2 and F4. Or it might not: it has to decide, at call 23, what will matter at call 31. Anthropic's guidance warns about exactly this: aggressive compaction can lose details whose importance only shows up later.
5. Notes kept all five facts at a quarter of the size
With five write_note calls, the agent kept every fact in its own short line, and clearing then removed the bulky tool output around them. The final prompt was 10,650 characters with 5 of 5 facts, 76% smaller than keep_all. The cost: 5 extra model calls (36 instead of 31) and $0.0624 instead of $0.0462, 35% more on this short run. On a run long enough to overflow the window, keep_all is not an option at all, and notes are what let the agent finish.
The lesson is not "use notes". It is that every strategy that shrinks the context decides what to drop, and the safest thing to keep is a short, explicit record of what the agent found. Whether a real model writes those notes reliably is the part you have to test with your model.
A context engineering checklist for your agent
- Log the actual prompts. Save the full text sent on every call of a few real runs and read them. Most context problems are obvious once you see call 40: the same file read three times, a 6,000-character JSON blob for a yes/no answer, the goal buried at the top.
- Make the prompt start stable, then only append. System prompt and tool list first and unchanged; no timestamps or user names at the top; serialise JSON with sorted keys. Check your cache hit rate in the API response. Our timestamp test shows what a single changing line costs.
- Return less from tools. Paginate, filter and summarise inside the tool. A tool that returns the 3 matching log lines instead of 3,000 is the cheapest context engineering there is. Chase's advice: "A short but descriptive error message will go a lot further a large JSON blob."
- Keep pointers, not payloads. Once a file or page has been read, a path or URL is enough to get it back. Manus designs every compression step to be "restorable" this way.
- Turn on tool-result clearing before anything fancier. It keeps the agent's reasoning and only drops bulky output. Set it to clear a meaningful amount at once, since each clearing breaks the cache from that point.
- Make the agent write down what it finds. A notes or progress file, or simply an instruction to restate key findings in its own words. In our test, this is what decided whether clearing lost facts.
- Compact late and check the summary. Write the compaction prompt to keep every decision, open bug and key number first, then trim. Test it by asking questions about early details after compaction.
- Isolate exploration. If a sub-task needs 50 file reads but its answer is one paragraph, give it to a sub-agent and keep only the paragraph.
- Measure with a fixed set of tasks. Change one context rule at a time and compare success rate, tokens per task and cache hit rate. Context changes feel safe and often aren't.
Built-in context tools in 2026
The techniques above no longer need to be hand-built. As of 21 November 2026, the main APIs and frameworks ship them, several still in beta:
| Platform | Feature | What it does | Status, 21 Nov 2026 |
|---|---|---|---|
| Claude API | Context editing: clear_tool_uses_20250919 | Clears the oldest tool results once the prompt passes a trigger (default 100,000 input tokens), keeping the last 3 tool uses by default; clear_at_least avoids breaking the cache for a small saving | Beta |
| Claude API | Compaction: compact_20260112 (at a threshold) and compaction on demand | Claude writes a summary on the server and the API drops everything before it; threshold compaction starts at 150,000 input tokens by default (minimum 50,000) | Beta (on demand since 14 Sep 2026) |
| Claude API | Memory tool | Claude reads and writes files in a memory directory that your code stores, so notes survive beyond the window | No beta header needed since 17 Feb 2026 |
| OpenAI Responses API | context_management with compact_threshold | Server-side compaction into an opaque compaction item carried into the next call | Available |
| OpenAI Agents SDK | Sessions: SessionSettings(limit=N), OpenAIResponsesCompactionSession | Keeps history across runs; send only the last N items, or compact long sessions automatically | Available |
| LangChain / LangGraph | SummarizationMiddleware, trimming with @before_model | Summarise or trim earlier messages before each model call | Available (replaces the older SummarizationNode) |
Anthropic reported in its September 2025 announcement that on an internal agentic search evaluation, context editing alone improved performance by 29% over baseline and context editing plus the memory tool by 39%, and that in a 100-turn web search evaluation context editing cut token use by 84%. Those are the vendor's own numbers on its own tasks; treat them as a reason to test, not a guarantee.
Every one of these features changes earlier parts of the prompt, so each one costs some cache reuse when it runs. Both Anthropic's and OpenAI's docs say so. That is the same trade-off our clearing and compaction rows show: smaller prompts, a cache that restarts from the first changed token.
Interview questions on context engineering
- What is context engineering, in one line? Choosing and maintaining the set of tokens the model sees on each call, so it has what it needs and little else.
- How is it different from prompt engineering? Prompt engineering writes the instructions once. Context engineering decides, before every call, what else goes in: history, tool results, retrieved data, memory. Prompt engineering is part of it.
- What is context rot? The drop in a model's accuracy at recalling and using information as the prompt gets longer, even well inside the context limit.
- Why can a timestamp in the system prompt raise costs? Prompt caches reuse only an exact prefix. A value that changes every call at the top of the prompt means nothing after it is reused, so every call pays full price.
- Compaction or tool-result clearing? Clearing first: it is lighter and keeps the agent's own reasoning. Compaction when even the reasoning no longer fits, accepting that the summary may drop details.
- When would you use sub-agents for context reasons? When a sub-task needs a lot of reading but returns a short answer. The reading stays in the sub-agent's window; the main agent only gets the summary.
- How do you know your context strategy works? Fixed evaluation tasks with known answers, run before and after the change, measuring success rate, tokens and cache hit rate. Also check the final prompts for the facts the answer needs, as in our test.
Honest limits of this test
- We measured what reaches the model, not the model's answer. No language model ran in this test. Each strategy's job is to put the right text in the prompt, and that part is exact and repeatable. Whether a model then finds the fix in a 44,533-character prompt, or in a 10,650-character one, depends on the model.
- The run is scripted, and the agent's words are ours. We decided that the agent would restate three facts and not two, before running anything. A real model might restate all five or none. The point is the mechanism: clearing and our summary keep exactly what the agent wrote down.
- Our compaction summary is a stand-in. It copies the agent's own lines and none of the tool output. A model-written summary reads the tool output and might keep F2 and F4, or might drop F3.
- The run is small. 30 tool calls and a 32,000-character trigger, where real agents run 50 to hundreds of calls and APIs trigger at 100,000 tokens or more. The ratios are what carry over, not the dollar amounts.
- Costs are estimates. We count about 4 characters per token; Anthropic notes its newer tokenizer produces about 30% more tokens for the same text, so real costs would be higher for every strategy. We assume a cache breakpoint at the end of each prompt, a 5-minute cache that never expires during the run, and that cached text is an exact prefix of the previous prompt. Output tokens are not counted.
- Quotes, prices and API names come from the sources below, read on 21 November 2026. Several context features are in beta and change quickly; check the docs before you build.
Frequently asked questions
What is context engineering in simple words?
It is choosing what the model sees on each call. An agent calls a language model many times, and each call needs a prompt built from instructions, tools, earlier messages, tool results and notes. Context engineering decides what goes in, what gets summarised or cleared, and what stays outside until it is needed.
What is the difference between context engineering and prompt engineering?
Prompt engineering is writing good instructions: the system prompt, the task and examples. Context engineering covers everything else that reaches the model too, such as history, tool output, retrieved documents and memory, and it is decided again by code before every call. Prompt engineering is one part of context engineering.
What is context rot?
Context rot is the drop in a model's ability to recall and use information as its prompt gets longer, even well inside the context limit. Chroma tested 18 models in July 2025 and found performance grew increasingly unreliable with input length; every model did better on a focused prompt of about 300 tokens than on the full one.
What are the main context engineering techniques?
Just-in-time retrieval (load files or data only when needed), tool-result clearing (drop bulky old tool output), compaction (summarise the history near the limit), structured notes (the agent writes findings to a file outside the window), sub-agents (isolate big sub-tasks) and keeping the prompt start stable so the cache works.
Why does prompt caching matter for agents?
Agents re-send their whole context on every call and write short replies, so input tokens dominate the cost. A cache reuses the unchanged start of the prompt at a fraction of the price. In our test, a timestamp at the top of the prompt dropped cache reuse from 94.1% to 0% and made the run 10 times more expensive.
Is compaction or tool-result clearing better?
Start with tool-result clearing: it only drops old tool output and keeps the agent's own reasoning, so it is the lighter change. Use compaction when even that no longer fits. Both can lose facts the agent read but never restated; in our test each kept 3 of 5 key facts, and adding agent notes kept all 5.
Do I need a framework for context engineering?
No. Each strategy in our test is a short Python function that turns the history into prompt text. Frameworks help when you want it ready-made: the Claude API has context editing and compaction (both in beta as of November 2026), OpenAI has server-side compaction and Agents SDK sessions, and LangChain has summarisation middleware.
Crio AI Engineering
Build agents that work outside a demo
Crio's AI Engineering program is project-first: you build LLM apps, RAG pipelines and tool-using agents, then learn to evaluate, secure and ship them. Talk to the team about whether it fits you.
Sources
- Effective context engineering for AI agents (Anthropic Applied AI team), 29 September 2025 · Anthropic
- Managing context on the Claude Developer Platform: context editing and the memory tool, 29 September 2025 · Anthropic
- Claude docs: Context editing (clear_tool_uses_20250919 defaults) · Anthropic
- Claude docs: Compaction (threshold and on demand) · Anthropic
- Claude docs: Pricing (Sonnet 5.5 input, cache write and cache read prices) and Prompt caching · Anthropic
- Context Engineering for AI Agents: Lessons from Building Manus (Yichao 'Peak' Ji), 18 July 2025 · Manus
- Context Engineering for Agents (write, select, compress, isolate), 2 July 2025 · LangChain
- The rise of "context engineering" (Harrison Chase), 23 June 2025 · LangChain
- Context Rot: How Increasing Input Tokens Impacts LLM Performance (Hong, Troynikov, Huber), 14 July 2025 · Chroma
- Lost in the Middle: How Language Models Use Long Contexts (Liu et al., arXiv 2307.03172) · arXiv
- Andrej Karpathy on context engineering, 25 June 2025 · X
- OpenAI docs: Prompt caching and Compaction · OpenAI
- OpenAI Agents SDK: Sessions · OpenAI
- LangChain docs: Short-term memory (trimming, SummarizationMiddleware) · LangChain
The test ran on 21 November 2026 with Python 3.12 and the standard library only; no language model was called, and the output is identical on every run. ShopKart's checkout bug and the agent run are invented. Prices, API names, statuses and quotes were read from the official docs, posts and papers on 21 November 2026.
Last updated: