Imagine you hire a brilliant employee with severe amnesia.
Every morning (in fact, before every single task) they re-read four things: their job description, their notes about the current client, the transcript of the conversation so far, and the list of equipment on their desk. Whatever isn’t on the desk does not exist for them. And whatever is on the desk, they will read, even if it’s irrelevant, and reading it costs time and focus they could have spent on the client.
That employee is your agent. Your job, as the context engineer, is to be the person who decides what’s on the desk. This chapter is the anatomy of the desk.
Anatomy of a single turn
Every turn of the conversation, the model sees a composite. Stack it up:
- The system prompt: static. Role, session flow, principles, tool usage rules. You write it; it changes only when you edit it.
- The memory injection: per session. A compact record like: “User: Ajitesh. Last session: classroom escalation, ended at step 3. Prefers examples over theory.” The platform (or your pipeline) writes it from previous sessions; Part 3 covers how.
- The conversation history: grows every turn. The transcript so far, possibly summarized beyond a threshold. The model and user write it together; the pipeline manages it.
- Tool definitions: mostly static. The schemas for the registered tools: image generation, MCQ, memory search, end-session. The platform writes these when you toggle tools on.
- Retrieved knowledge: on demand. Chunks pulled from the knowledge base when the agent asked for them last turn.
That composite is “the context.” When builders say an agent “knows” something, they mean it is somewhere in this stack. When an agent “forgets” something, it fell out of this stack. Every behavior and every failure routes through these five components, which is what makes the anatomy worth memorizing.
Refresh cadences
The organizing insight of workspace design: information belongs at the cadence it changes.
- Never changes during the product’s life (role, personality, principles) goes in the system prompt.
- Changes per user or per session (name, progress, preferences) goes in the memory injection.
- Changes per turn (the conversation itself) is history, managed automatically and summarized when long.
- Needed occasionally and unpredictably (manuals, price sheets, protocols) is referential: knowledge base or just-in-time tools.
Misfiled information is the root cause of most agent bugs. Put the sales manual in the system prompt and you pay for it on every turn of every session: bloat that dilutes attention exactly as Chapter 5 warned. Put the agent’s core personality in the knowledge base and you get a coach that forgets who it is unless it happens to search. Neither bug looks like a “filing” bug from the outside; both are.
When you find yourself unsure where something goes, ask the cadence question: does the agent need this every turn, this session, or only when asked? The answer is the filing decision.
The cost of every token
The economics deserve to be concrete, because they are larger than intuition suggests.
A 10-minute voice session runs roughly 60 to 100 turns. The system prompt is re-processed on every one of them. A 5,000-token system prompt therefore costs on the order of half a million system-prompt tokens per session, before the user says anything substantive. That is the “token tax”: every word in the prompt is billed 60 to 100 times per session, forever.
Cutting prompt bloat in half cuts a real slice of your per-minute cost. And (this is the part that makes the discipline free) it typically improves behavior at the same time, because of Chapter 5’s accuracy argument. Leaner context is cheaper and better. There is rarely a trade-off to agonize over; there is just editing to do.
Tip: make it a monthly ritual. Paste your full system prompt into a token counter. For each section, ask one question: does the agent need this on literally every turn? If not, it is a candidate for the knowledge base (Chapter 9) or a tool instruction (Chapter 8). Most prompts shrink 30 to 50% on their first audit with zero behavior loss.
Who assembles the context?
The composite has to be assembled fresh every turn, and who does that work is platform-dependent, so it is worth demystifying so nothing feels like magic.
On an experience platform, a builder agent (a meta-prompter) first compiles your natural-language requirements into the workspace: a generated system prompt, selected and configured tools, inferred tool data, memory configuration, and guardrails. Then at runtime, a context manager assembles each turn’s composite (prompt plus memory plus history plus tools plus retrievals) and handles summarization when the history grows long.
On Tough Tongue: describing a scenario in plain English triggers exactly this pipeline: the platform generates the
ai_instructions(the system prompt), registers tools, and wires memory. You can then open the scenario and edit every piece directly, which matters: Chapter 3’s context-control axis is about having this door open. The Writing AI Instructions guide documents the format the builder agent produces.
On DIY stacks, you write the assembly loop yourself, which is more work but the same design. Either way, the decisions are yours: what goes where, at which cadence. The platform can automate the assembly; it cannot know your domain.
The rest of Part 2 takes the surfaces one at a time. First, the highest-leverage two thousand words you will write: the system prompt.