AI Tutors Can Teach. Who Will Learn?

AI is no longer just acing exams. A growing number of benchmarks suggest it is getting better at helping people do that too, sometimes as effectively as human tutors.

In September, Handshake unveiled StudentBench, a suite of AI teaching evaluations and a public platform with more than 175,000 student–AI messages. In an accompanying study of over 2,000 adults who spent an hour studying for the GRE, researchers found that AI tutoring produced learning gains similar to human tutoring. By Handshake’s estimate, the most cost-effective AI tutor can run about half a cent per point of improvement, versus nearly $5 for a human tutor charging $75 an hour.

A month later, humans& introduced Ampersand Tutor, trained against a simulated learner that was benchmarked against real student data, including Handshake’s. In the company’s testing, under a plain prompt that simply asked each model to tutor well, Ampersand Tutor outperformed leading models from OpenAI and Anthropic. 

Suffice to say: evidence is accumulating that AI tutors can teach effectively under the right conditions, at least for those who opt in and want to learn. Yet in everyday life, motivation is a harder problem. Most benchmarks don’t measure it, or whether AI tutors can understand a learner over time. That gap between teaching well in neat, controlled settings and helping students learn in the messy, real world is the next frontier.

Motivation: Edtech’s Field of Dreams

When Stanford researchers studied elementary students who were given an AI reading tutor, nearly half never logged on. The problem is hardly unique to AI. Education technology has long wrestled with a “Field of Dreams” problem, that just because someone builds it, students will come. In his analysis of studies on popular digital math programs, Laurence Holt found that only 5 percent of students used them as recommended. The other 95 percent who did not saw “minimal gains, if any.”

Across a decade-plus of investing in learning tools, we’ve seen that the ones that deliver strong outcomes rarely do it on the strength of tech alone. Behind them is a teacher, coach, parent, or some external goal that gives students a reason to care, alongside routines and support that help them follow through. In the same Stanford study, when researchers paired students with a coach, pupils read as much as 80 percent more stories on the platform. The quality of an assignment, coupled with a student’s relationship with a teacher, matters as much as any product feature. Together, they influence whether students do the hard work or simply seek shortcuts.

Motivation is what’s largely missing from current AI tutoring benchmarks. Most grade tutors on whether explanations are correct, whether they give scaffolded hints instead of answers, and how well they can adapt when a learner is confused. Of the open benchmarks we’ve reviewed, few measure how well a tutor keeps learners engaged.

Part of the reason is that engagement is easy to treat as someone else’s job — a teacher, parent, or coach. But expert tutors embed it into the experience. When Polygence analyzed its own one-on-one human tutoring sessions, rapport-building took up as much as 15 minutes of the first hour. Noticing frustration, knowing when to push or ease off, and keeping a student talking are intangible, invaluable tutoring skills.

The same blind spot shows up in pedagogical soundness, which benchmarks are beginning to measure. AI models are far better at answering and explaining questions than at anticipating the many ways students may get something wrong. Great human tutors can pinpoint learning gaps, past struggles, and nonacademic cues, and carry this picture into the following week’s session. They can adapt their style, inject humor, let silence stretch, and use other subtle moves to engage their students. For an AI tutor to do the same takes authentic, multimodal data from real tutoring sessions, which is scarce and hard to synthetically produce. James has described this as AI’s gray area problem.

Today, AI tutoring benchmarks are largely limited to text-based data that come from a single conversation or session. When researchers do stretch the conversations out, cracks appear. In one benchmark, AI tutoring mishaps — like handing over answers or reinforcing student misconceptions — jumped from 18 percent in a single exchange to nearly 78 percent over a long conversation. Another benchmark found that when cheating requests were disguised as homework help, models played along more than 90 percent of the time.

The Test for AI Tutors and Their Builders

For the next generation of AI tutors, the real test is understanding motivation, why students make mistakes, and which teaching tactics actually lead to student learning. Founders building them should be prepared to discuss these questions:

1. Who are your learners? Be honest. It’s unlikely that a startup learning tool can serve every student from the get-go. Self-directed, driven learners are a well-established market. But there are many others who need help staying motivated. English language learners and those with different neurodivergent profiles are not well served by existing services and materials. Where will you focus, and where will you expand?
   
   
2. How will you reach them? In school, after school, or at home? Through a district, a parent, or a learner paying their own way? The channel shapes whether students show up, and how fast and far your tool can grow.
   
   
3. How do you know your tutor teaches well? What is your rubric for good pedagogy, and does it hold up past the first exchange? Explicit instruction, spaced and interleaved practice, and retrieval are as well supported as Socratic questioning, and “never give the answer” is a cruder rule than what good tutors usually follow. Does the tutor remember what the student struggled with last time, or bring earlier material back at the right moment? Can it exude the energy, playfulness and other characteristics of great human tutors?
   
   
4. Where does the learning data come from? What longitudinal interactions — from tutors, classrooms, families, or others — are being captured to train and improve the tutor? Or are you relying on public datasets and benchmarks that anyone can use? Do you have access to multimodal data (audio and video) from real sessions that reveal behaviors that text transcripts miss?
   
   
5. How are you measuring outcomes? What are your evals, who wrote them, and what real student interactions are they based on? Beyond the session, do students come back, persist after a wrong answer, and retain what they learned weeks after? 
   

If you’re building AI tutors with concrete perspectives on any of these questions, we’d love to hear from you!

Current AI Tutoring Benchmarks

Below is a compilation of open benchmarks as of November 2026.

Thank you to Jennifer Carolan and Susanna Loeb for reading earlier drafts of this piece.