The craft of building with AI has been redefined three times in three years.
In 2023, AI entered the mainstream, and the core craft was prompt engineering: experimenting with exactly what to say, what context to pack into a single prompt, to get the model to act. One input, one output.
In 2024, tools arrived. Tell an agent to take an action, implement the tool, and it acts. Now the craft was the interaction between the prompt and the tool instructions, two things that must agree with each other, written in two different places.
Then builders started assembling whole agents. Something goes in the system prompt. Other things live in a knowledge base the agent reads on demand. The agent has tools. It can reach external tool servers for capabilities nobody wired by hand. It remembers users across sessions. At that point you are no longer writing a prompt. You are designing a workspace for a language model to operate in.
That is context engineering: not what you say to the model, but everything you arrange around it. It is the core skill of this playbook, and this part of the book teaches it surface by surface.
If you prefer a video introduction to the topic, this session covers the same ground:
Why bother, when context windows are huge?
The obvious objection first. Context windows have hit a million tokens, with larger versions in the labs. Why not stuff everything in and let the model sort it out?
Because accuracy is consistently higher with less, more relevant context. Research and practice agree on this: models degrade with long, diluted context. The “lost in the middle” effect is real: facts buried in the middle of a long context get missed. And the mechanism is architectural, not a bug to be patched: the transformer pays attention to every token you include. Everything in the context either helps the current turn or dilutes it. There is no neutral content.
The practical rule: if you are really careful about what you put into the agent’s environment, you are more likely to achieve a better outcome, even though you could stuff far more in.
A quantitative anchor to hold onto: beyond roughly 10,000 tokens of system prompt (about 7,500 words), you are in diminishing-returns territory. But the qualitative bar governs before the quantitative one does. Everything present must earn its place, at any size.
The reference demo, reverse-engineered
Return to Chapter 1’s coaching demo, because now you can dissect it. Every behavior the agent displayed traces back to a specific surface of its workspace:
| Observed behavior | Workspace surface that produced it |
|---|---|
| ”Hi Ajitesh, great to have you back” | Memory: a compact summary injected into the system prompt |
| ”Today we’ll dive into another classroom scenario” | Memory (last-session record) plus the session flow in the prompt |
| Paints a picture of the classroom | Image generation tool, plus tool instructions in the prompt |
| Presents a multiple-choice card | MCQ tool, plus a flow step that calls for it |
| ”Let’s go deeper. What’s your second step?” | Coaching principles in the prompt: push two to three layers deep |
| Stays a warm, structured coach throughout | The role and character section of the prompt |
Nothing on the left is magic. Everything on the left is a line someone wrote on the right. That correspondence, behavior you can point to in source, is what a well-engineered agent looks like, and it is the standard the rest of this playbook holds you to.
The agent equation, upgraded: an agent equals reasoning (the LLM) plus tools (actions) plus state (memory and history). Context engineering is the practice of designing all three, not just prompting the first.
The four surfaces of the workspace
Everything you will arrange around the model falls into four surfaces. This list is the map for the rest of Part 2 and Part 3:
- The system prompt: identity, session flow, principles. Present on every turn, re-read on every turn. Chapter 7.
- Tools: the registered actions, plus the instructions for when and how to use them. The instructions matter more than the registration. Chapter 8.
- Referential knowledge: knowledge bases, documents, external tool servers. Fetched just in time, only when needed. Chapter 9.
- Memory and state: the conversation history, cross-session memory, and summaries. Part 3.
And above all four sits the meta-decision that Chapter 10 formalizes: what lives permanently in the context versus what gets referenced on demand. Get that split right and everything else gets easier; get it wrong and no amount of prompt polish compensates.
One more thing the list makes clear: when your agent misbehaves, the question is never just “what should I add to the prompt?” It is “which surface owns this behavior?” A fact the agent doesn’t know might belong in knowledge, not the prompt. A tool it never uses needs an instruction, not a bigger schema. A user it forgot needs memory, not apologies. Chapter 21 turns this into a full debugging table, but the habit starts now.
We will start with the surface you control most directly and misuse most often: the system prompt. But first, the next chapter assembles the whole workspace in one picture, so you can see what the model actually reads on every single turn.