Chapters

Part 1 · Chapter 4

What AI changes, and what it doesn't

The cost math, the death of the leakable exam, and the parts of teaching that stay stubbornly human.

8 min read · Updated Jul 2026

What you'll learn

  • The real per-student economics, with numbers from a deployed university exam
  • Why rubric-generated conversations have nothing to leak
  • How the professor's role relocates from performer to designer and debriefer

Before mapping use cases, it is worth being precise about what AI actually changes in educational role play. Three things change fundamentally. Two do not change at all, and pretending they do is how programs fail.

Change 1: The economics invert

The clearest public data point comes from NYU’s Stern School of Business, where professor Panos Ipeirotis ran AI-conducted oral examinations for an undergraduate-level AI course: 36 students, roughly 25 minutes each, personalized to each student’s capstone project. Total platform cost: about $15, or $0.42 per student.

Run the counterfactual honestly. A human examiner conducting 36 sessions of 25 minutes, plus scheduling overhead, plus grading, is multiple full days of faculty or TA time; at teaching-assistant rates it is hundreds of dollars, and at faculty opportunity cost far more. An actor-based equivalent is thousands. An avatar-platform equivalent, at $49 to $164 per session, is $1,800 to $5,900.

The consequence is not “the same thing, cheaper.” When a cost drops three orders of magnitude, the activity changes category. Ipeirotis’s framing is exactly right: at $0.42 per student, an oral comprehension check no longer needs to be a once-a-semester final. It can be attached to every assignment. Role play stops being an event you ration and becomes infrastructure you assign, the way problem sets are assigned. Budget-constrained programs, which is all of them, should notice that this moves conversational practice from the “special initiative requiring a grant” column to the “normal course activity” column.

Change 2: There is nothing to leak

Traditional assessments face a security dilemma: the more students know about the exam, the less it measures. Question banks leak. Last year’s students brief this year’s. Standardized-patient cases circulate in group chats. So exam content is hoarded, which means students cannot practice the actual thing being assessed.

A rubric-driven AI conversation dissolves the dilemma. When the examiner generates its questions dynamically from a rubric and from the student’s own submitted work, there is no fixed exam to leak. The NYU team published the entire examination structure to students in advance, on purpose. A student who wants to prepare has exactly one option: get better at explaining their own project and reasoning through cases. Practice is learning. The preparation and the competence are the same activity.

This property extends beyond exams. A role-play scenario whose counterpart improvises within a persona cannot be memorized the way a script can. The student who has run the “skeptical dissertation committee member” scenario ten times has not learned the answers; they have learned to think on their feet, because the tenth conversation was not the first one repeated. Chapter 12 builds on this: when the breakthrough conditions mirror the rubric, gaming the exercise and mastering the skill become the same thing.

Change 3: Every session becomes evidence

Human role play evaporates. It happens, it ends, and what remains is memory and maybe an observer’s checklist. AI role play leaves a complete artifact: recording, transcript, timing, rubric scores. That artifact enables things that were previously impossible: a student comparing their attempt one against their attempt nine, an instructor seeing that the whole cohort collapses at the same conversational moment, an accreditor receiving practicum-hour logs that are actually verifiable, an appeal process for a contested grade that consults the recording instead of two memories. Chapter 18 is built entirely on this property.

What does not change: the debrief

Now the other side of the ledger. Every pedagogical source in Chapter 2 agrees the debrief is where experience becomes learning, and no part of that is automated away by making the performance synthetic. A student who runs fifteen AI sessions and never reflects on them has exercised, not learned. Worse, high volume without reflection can entrench bad habits at scale, the same way unsupervised batting practice can groove a bad swing.

AI can assist reflection (Chapter 13 covers transcript-anchored feedback and “chat with your feedback” patterns), but the classroom debrief, the reflection memo, the moment a professor connects a student’s fumbled answer to a concept from week three: that is human work, and the abundance of practice makes it more valuable, not less. Programs that treat AI role play as a content library to assign and forget consistently plateau. Programs that treat it as the practice layer feeding a human coaching layer compound.

What does not change: design is the product

The second stubbornly human thing is Stage 1. An AI counterpart is exactly as good as the scenario behind it: the persona’s motivations, the resistance calibration, the breakthrough conditions, the rubric. Those come from domain expertise, and the domain expert is you. The professor who has watched two hundred students fumble the same moment in a counseling intake knows precisely what the synthetic client should do at minute four. No model knows that out of the box.

This is good news, professionally. The fear that AI role play deskills teaching has it backwards: it removes the parts of the job that were never the expertise (performing the angry parent for the fortieth time, scheduling actors, proctoring) and concentrates the job in the parts that were (knowing what a great conversation looks like, designing the situation that teaches it, coaching the attempt). The professor relocates from performer to designer and debriefer. Part 3 of this playbook is the craft manual for that new job.

Exercise

Compute the per-student cost of your current approach to conversational practice or oral assessment, honestly: include actor fees or platform session fees, but also faculty and TA hours at a realistic hourly rate, and scheduling overhead. Then write down what you would assign if that number were $0.50. Not what you would do to the current activity — what you would assign, per week, if rehearsal were effectively free. That second list is your candidate roadmap for Part 2.