In a randomized trial spanning 19 universities and detailed in a preprint released November 14, students in applied STEM courses achieved higher scores on proctored exams after their instructors introduced course-specific AI tools, compared with peers in classes where the tools had not yet been deployed.
Led by Jun Li of the University of Michigan, the study estimated a student-level improvement of 0.32 standard deviations, with a 95% confidence interval ranging from 0.06 to 0.59.
That gain equals roughly one-third of the usual variation in scores within a single class, though the precise magnitude remains uncertain. To compare results across courses using different exams, researchers standardized scores within each class.
The preregistered trial enrolled 7,200 students across 55 applied STEM classes over three semesters, running from spring 2025 through spring 2026. Each course-and-instructor combination was treated as a single class.
Exam analysis focused on 4,768 students across 44 course offerings. Researchers incorporated results from prior terms to control for pre-rollout class performance.
The exam analysis was limited to proctored, in-person tests—mostly paper-based. Other exams were administered on strictly monitored or restricted-access computers at university testing centers.
This screening rule was added after preregistration, and course offerings where exams might permit AI assistance were excluded.
Relative to delayed rollout, the estimated student-level gain was 0.32 standard deviations (95% CI 0.06 to 0.59), using earlier exam results as a baseline.
The trial focused on applied STEM classes. The exam-security filter was added after preregistration, and the timing comparison does not isolate which tool designs worked best.
How Instructors Used AI in Their Courses
Researchers grouped classes by subject and academic level, then randomly assigned them to receive the tools either immediately or one teaching term later.
Instructors in both groups received design and technical support, including training sessions, examples of tool instructions, and access to a platform for building and deploying the tools.
After instructors introduced the tools, all students in those classes could use them during the term, prior to taking the exams used to measure learning.
Some instructors developed tutors that provided targeted practice and feedback, while others created coaches that guided students through problems by asking questions. Additional instructors built interactive simulations and role-play exercises in which students interrogated AI-generated stakeholders.
The study found no statistically significant change in engagement or satisfaction. Engagement metrics varied by class and included attendance and assignment completion, while satisfaction was measured through end-of-term course evaluations.
Because the trial compared rollout timing, it does not show how course-specific tools would perform against generic chatbots. The extent to which each design contributed to the gains, and whether the results apply beyond applied STEM, remains uncertain.

