Picture two kids working through the same math unit. Both use a chatbot to help with homework. One breezes through every practice problem and turns in a flawless worksheet. The other struggles through the same problems more slowly, makes mistakes, and asks for help twice.
A month later, they sit the same closed-book test — no chatbot allowed. The one who breezed through practice does worse. Not a little worse: up to 17% worse, according to a large randomized study cited in the OECD’s Digital Education Outlook 2026, a 245-page report released this January.
The study wasn’t a hunch or a survey. It was a controlled trial with high school students in Türkiye. When students got access to GPT-4 for math practice, their practice scores jumped by as much as 127%. Then, on the exam that actually counted, those same students scored worse than peers who’d studied the material without any AI help at all.
The OECD has a name for that gap: the "mirage of false mastery." Homework looks done. Practice scores look great. But the thinking homework is supposed to build never actually happened.
Worth being precise about scope here: this is one (large, well-designed) field experiment in one subject, cited as evidence inside a much bigger report — not a claim that every use of AI tutoring produces this exact result. But it’s the clearest number-backed version yet of something we’ve been saying since Auxesis started.
Same enemy, new evidence
We’ve called this the illusion of mastery from day one — a good grade, a clean worksheet, or a nodding head that hides the fact that real cognition never occurred. It’s the thing this company exists to fight. Now an intergovernmental body, working from research across dozens of countries, has independently landed on the same conclusion, in almost the same words.
Why the gap happens
The mechanism isn’t mysterious. When a student hands a problem to a generic chatbot, the tool does the diagnosing, the reasoning, and the checking — the exact steps that turn information into understanding. The student’s job shrinks down to reading an answer and typing it in. The OECD calls this "cognitive offloading," and its data says it’s the default mode most students fall into with off-the-shelf tools, because it’s faster and it feels productive.
The report’s fix isn’t "ban AI in the classroom." It’s a shift from what it calls "fast AI" — tools that hand over a finished answer — toward "slow AI": tools designed to keep a student thinking, questioning, and revising, with a teacher or parent still steering. It also calls for assessment to catch up: instead of grading only the final answer, look at the drafts, the reasoning, the back-and-forth that produced it.
What this means if you’re homeschooling
If your kid comes home with a stack of perfect homework and a chatbot did most of the thinking, you won’t see the gap until the test — by which point it’s expensive to fix. The fix isn’t banning the tool. It’s one question before you accept the answer: "Walk me through how you got this." If your child can’t, the homework was a mirage, not mastery. It’s also exactly why depth beats speed at home — a slower, more supervised pass through a concept builds something a fast chatbot answer can’t. (This is the whole premise behind how we teach Singapore Math in the Parent Track.)
What this means if you’re running a program
If you’re a microschool, co-op, or tutoring program evaluating AI tools, this report is a warning about your stack. A generic chatbot bolted onto a curriculum will likely produce this same performance-learning gap at scale — and you won’t see it until standardized scores come in. Process-oriented data instead of product-only grading, human judgment kept central, AI that scaffolds rather than substitutes: that’s close to a working blueprint for how an AI layer belongs in instruction, and it’s the design logic behind COMPASS.
The takeaway
An intergovernmental body just ran the numbers and landed where educators who care about real learning have been standing for years: a good-looking answer isn’t the same as understanding, and the fastest path to a finished worksheet is often the slowest path to actually learning the material. The struggle isn’t the obstacle. It’s the point.
Leave a Reply