A follow-up to The 17% Tax — same research family, different question.
We wrote recently about the OECD’s finding that AI-assisted math practice can look great and still leave nothing behind — up to 17% worse performance once the AI is taken away. That post ended with one diagnostic question: “walk me through how you got this.”
There’s a second piece of evidence worth its own post, because it answers a question the math study doesn’t: how fast does the gap open, and does it show up outside of math?
The one-hour test
A study cited in the same OECD Digital Education Outlook 2026 had students across several US universities write a short essay — one group alone, one with a search engine, one with a general-purpose chatbot doing much of the drafting. One hour later, researchers asked each student to quote a sentence from what they’d just “written.” Among the unaided and search-engine students, 89% could. Among the chatbot group, only 12% could.
Worth being precise here, in the same spirit as the caveat we ran last time: the specific study behind this appears to be MIT Media Lab’s “Your Brain on ChatGPT” research (Kosmyna et al.) — a small trial (54 participants), still a preprint, not yet peer-reviewed. Different write-ups of it report slightly different numbers (some cite 90%/17% instead of 89%/12%), which is normal for early-stage research moving through secondary coverage, but it means this shouldn’t be treated as a settled, precise figure — just a strong, repeatable signal in the same direction as the math result: fast AI produces work that looks finished but was never really held by the student who “wrote” it.
One hour. Not a semester, not a unit test — sixty minutes was enough for four out of five students to lose their grip on their own sentences.
Why one question isn’t enough for this one
The “walk me through how you got this” check works well for a worked math problem because there’s a step-by-step path to retrace. Writing doesn’t hand you that same rope. A finished essay doesn’t show its work the way a solved equation does — which means the single-question check from the math post genuinely won’t catch this failure mode. You need something with more structure.
That’s what Auxesis’s Metacognition framework is for — three questions, asked in sequence, that work regardless of subject:
- Monitor — “Before you turn this in: what’s the one sentence in here that’s most you? Point to it.” If your child can’t find one, that’s the tell — not a bad grade, just a flag that something outside their own head produced the words.
- Regulate — “What would you do differently if you had to write this again without any help?” This isn’t a punishment question. It’s the moment that turns a flagged gap into an actual second pass — the regulation step is what separates “I noticed this wasn’t mine” from “I fixed it.”
- Evaluate — Circle back a day or two later, unannounced, with a version of the one-hour test: “Explain the argument you made in that essay.” If they can’t, you’ve learned something real about the assignment — and caught it while it’s still cheap to address, not at the next test.
Why this can’t be a one-time fix
The math study and the essay study point at the same underlying mechanism from two directions: performance during the task tells you almost nothing about what’s going to stick. The only way to know is to check after — which is exactly what Monitor/Regulate/Evaluate is built to do, and exactly what a finished worksheet or a polished essay can’t tell you on its own.
This is also why it’s a framework and not a one-off question: the math post’s single check catches one failure mode; a repeatable three-step loop catches it across subjects, because “did my child actually think this through” isn’t a math-specific problem — it’s the same one whether the tool wrote an equation or a topic sentence.
If you want the full walkthrough of how we teach Socratic questioning at home — the same instinct that powers Monitor/Regulate/Evaluate — that’s covered in the Parent Track.
Leave a Reply