Sheet 08 of 17 · Outcomes
Learning outcomes
Effects depend heavily on design. Scaffolded tutoring helps; unrestricted answer access can raise assisted scores while lowering unaided ones.
Table 8.1: Four outcomes, measured separately
Better work with AI is not the same as learning. Each study below is sorted by what it actually measured. Only column D shows whether students can do the work on their own.
A Assisted performance Quality of work produced with AI Details +
B Speed Time taken to complete the task Details +
C Perceived learning How much students feel they understood or engaged Details +
D Independent competence Performance without AI, ideally after a delay Details +
| Study | A · Assisted | B · Speed | C · Perceived | D · Independent |
|---|---|---|---|---|
| Bastani et al. 2025 · RCT | +48% general GPT; +127% tutor | — | — | −17% after general GPT; tutor largely avoided the loss |
| Kestin et al. 2025 · RCT | — | Less time on task | Higher engagement and motivation | More than 2× median gains on post-test |
| IZA 2025 · experiment | — | — | — | +0.23 SD exam score (check whether AI was available) |
| Middlebury · experiment | — | — | — | +0.27 SD knowledge test; more durable for augmentation |
| Chen & Cheung 2025 · meta-analysis | Mixed; not separated | — | g = 0.62 affective-motivational | Not separated from assisted results |
— = not measured or not reported. Classification is this dashboard's reading of each study; check the originals.
Fig. 8.2: Effect sizes by outcome (Hedges' g)
0 0.5 1.0 1.5 2.0 2.5
Fig. 8.3: Assisted gains vs. unaided performance
Change relative to control, ~1,000 high-school maths students in Turkey. Assisted practice improved; once AI was removed, the general-chatbot group did worse. Tutoring safeguards largely prevented the loss.
RCT · Harvard physics · n=194 ↗
>2×
median learning gains from a pedagogically scaffolded AI tutor vs. active-learning class, in less time.
Details ++0.23 SD
test performance from AI tutoring; strongest for low prior knowledge and strong self-regulation.
Details ++0.27 SD
immediate knowledge scores; more durable for augmentation than automation users.
Details +Reading note: Meta-analytic effects come mostly from structured interventions, not everyday chatbot use. Separate four outcomes: a better answer with AI, faster completion, feeling that one understands, and retained competence without assistance.