Two-tier memory architecture

Here is a puzzle from the demo. The agent greeted Ajitesh by name instantly: no tool call, no pause, the name was simply there in the first second of the session. Yet minutes later, when a specific detail from three sessions ago came up, the agent paused briefly before recalling it.

Instant and delayed recall in the same conversation. That is not an inconsistency. Those are two different tiers of memory doing two different jobs, and neither tier could do the other’s job well.

The two tiers

The architecture in one sentence: the agent decides what to remember about you and puts a compact version in the system prompt (call it super memory) while all the details go into referential memory that the agent can query just in time.

You will recognize the shape: this is Chapter 10’s hybrid pattern applied to memory. Summary resident, detail on demand.

Tier 1: super memory, in the prompt. Injected into the system prompt at session start, so it is present on every turn with zero retrieval latency, which is why the greeting was instant. Contents: the three defaults from Chapter 11 (last-session summary, preferences, what’s next) plus your domain-specific facts, compressed hard.

An example injection:

## USER MEMORY
Ajitesh: 4th session. Working on classroom escalation strategies.
Last session: handled Mia's off-topic interruptions; chose acknowledge-and-boundary;
struggled with step 2 when the gentle reminder fails. Prefers concrete examples over theory.
Next: continue the escalation ladder; introduce parent-communication scenario if time allows.

The budget: a paragraph, not a page. Super memory pays Chapter 6’s every-turn token tax on every future session, forever. A bloated super memory is a permanent tax increase on the whole relationship.

Tier 2: referential memory, the searchable store. Full session summaries, key exchanges, accumulated facts: everything too bulky for the prompt. Accessed via a memory-search tool, which costs Chapter 9’s 300 to 500ms just-in-time latency. That is why the old detail came back after a pause, and why good agents search at natural moments rather than mid-sentence.

Because memory search is a tool, Chapter 8’s three questions apply verbatim. When to use it: “when the user references a past session or detail not in your user memory.” How to interpret: “weave the facts in naturally; never recite them back like a database.” How often: sparingly. A session that searches memory every turn feels like talking to someone reading your file in front of you.

Who writes memory? The agent does

Here is the write path most builders never think about: nobody types these memories in. At session end, the agent (or a background summarizer) produces the memory record: what happened, what to remember, what’s next.

You steer that writer with memory instructions, a configuration surface as important as the prompt itself: “Remember the user’s stated goals, their strongest and weakest skills demonstrated this session, and any personal context they volunteer.” Chapter 11’s heuristic becomes an instruction: tell the agent to note what a human expert in your domain would note.

Then trust, but verify. Read the actual memory records during your iteration loop (Chapter 21). Left unsteered, agents reliably over-remember trivia (the exact wording of an exercise, the timestamp of a digression) and under-remember feelings: that the user got frustrated, that they lit up at a particular win. The feelings are usually the more valuable record, and they only get captured if your memory instructions ask for them.

On Tough Tongue: memory is a per-scenario toggle (memory.is_memory in the scenario config). On, the platform records the standard fields (session summary, preferences, next steps), and the memory-instructions box lets you add domain-specific directives on top. Super memory injection and the referential store (searched via the memory_search tool from the tool reference) are handled by the platform. Your job is the directives.

Privacy and expectations

An agent that remembers people carries obligations, and they are cheap to meet if you meet them early:

• Tell users what the agent remembers. The greeting itself can do this naturally: “Last time we worked on your escalation ladder” is both continuity and disclosure in one sentence.
• Provide a reset path. Platforms typically expose memory clearing; make sure your users can find it, and mention it when someone seems surprised by what the agent knows.
• Don’t memorize sensitive details you don’t need. Health, finances, relationships: unless your product is explicitly about them, memory instructions should say what not to record. Negative memory rules count against Chapter 7’s 10 to 15 negative-rule budget, so spend them on the categories that matter.

Memory solves continuity across sessions. But even within a single long session, the context keeps growing every turn, and at some point, something has to give. That is context window management, next.