For most creative scenarios, you will not get it right on the first attempt. Follow the patterns in this playbook and you will get 80% of the way there. The rest arrives one transcript at a time.
Picture a builder’s fourth day with her memoir interviewer. Three complaints from testing: it talks too much, it forgot the user’s sister from session one, and it once ended the call abruptly at minute six. Three bugs. And, it will turn out, three different layers. Talks too much: a missing principle in the prompt. Forgot the sister: a memory-instruction gap. Abrupt ending: an end-session tool with no frequency line. By evening, three one-line fixes, each in a different file of the workspace.
That diagnostic instinct (this bug lives in that layer) is the skill this chapter installs, wrapped in a loop that makes it routine.
The loop, formalized
- Run a full session. Not two exchanges but the whole flow, opening to close. Bugs cluster at the seams and the late minutes, exactly where short tests never reach. Three or more full sessions before any verdict.
- Read the transcript cold. Ideally hours later, when you read what the agent said instead of what you meant it to say. Mark the single worst moment. Not all of them. The worst.
- Diagnose the layer. Which workspace surface produced this moment? The table below.
- Make one change. One. Multi-change iterations feel faster and are slower: when the next session improves, you won’t know which change did it, and when it regresses, you won’t know which to revert.
- Re-run the same situation. Same opening moves, same curveball. A fix you can’t reproduce against isn’t verified; it’s hoped for.
- Log it. Date, symptom, change, result. One line. Then loop.
Expectation-setting, so nobody quits at iteration four: about ten iterations to a good conversation. Under an hour to a working one, a weekend to one you’re proud of.
Diagnosing the layer
The debugging table this playbook has been building toward, symptom on the left, surface on the right:
- Wrong personality, breaks character, caves too fast → system prompt: the role block or the resistance block (Chapters 7, 18).
- Never uses a tool, or overuses one → tool instructions: a missing trigger or frequency line (Chapter 8).
- Makes up facts it should know → knowledge: content missing from the knowledge base, or its catalog line missing from the prompt (Chapters 9, 10).
- Forgets things across sessions → memory: instructions too vague, or the fact belongs in super memory rather than the referential store (Chapters 11, 12).
- Blurs or repeats itself late in long sessions → context management, or the session has outgrown one agent (Chapters 13, 14).
- Feels different after a phase change, or doesn’t when it should → flow transitions or handoff notes (Chapters 16, 20).
- Gets hijacked or drifts off scope → guardrails (Chapter 19).
- Mispronounces numbers, names, formulas → usually the model layer, not the prompt. Try a different TTS or voice before writing prompt rule number sixteen (Chapter 2).
The meta-skill sits above the table: resist the reflex that every bug is a prompt bug. Half of them live elsewhere, and prompt patches for non-prompt bugs are exactly how 40-rule prompts happen. The patch doesn’t fix the bug, so another patch follows, and the prompt grows scar tissue.
Test like a user
Builders unconsciously feed their agent the inputs it handles well. Three countermeasures:
- Play the persona honestly. Answer the way your real user would, including badly. Dedicate one full session to being deliberately lazy, vague, and off-topic. That single session tests Chapter 18’s calibration and Chapter 19’s guardrails in one pass.
- Recruit two outsiders before calling it done. Most remaining issues arrive within minutes of the first outside tester, because outsiders don’t know which questions the agent is good at.
- Test the seams. Quit mid-session and return: does memory hold? Run long: does summarization keep the right facts? Interrupt constantly: does barge-in recover? Pick wrong MCQ answers on purpose: does the agent actually adapt, or barrel on?
Tip: version your prompt like source code (Chapter 3’s portable-assets rule, now operational). A dated change log, one line per iteration: “v7: deepened resistance layer 2; users were winning too fast.” It is the cheapest infrastructure you will ever build, it is the rollback path Chapter 24 depends on, and it is the difference between iterating and thrashing.
On Tough Tongue: every session produces a transcript and, when enabled, a rubric-scored analysis automatically: the raw material of this loop with zero setup. Sessions and their evaluations are also retrievable programmatically via the sessions API, which Part 7 uses to run this loop at production scale.
The loop tells you when a tool instruction is broken. The next chapter is the catalog of tools worth instructing in the first place, and the test that decides which ones stay.