The Wrong Training for the Wrong Problem

This week I sat down at my laptop and tried to imagine being a student. I opened the Gemini app that now lives in my computer's menu bar and told it I needed to write an essay about the Civil War. It attempted to introduce some friction and asked three sensible questions about my thesis, my format, and my progress. I ignored them and asked for a good thesis about racism. It gave me four. I said option four (Racial Attitudes in Military Service) was good and asked for three supporting examples. It gave me three, each with its connection to the thesis already written out. I asked for a five-paragraph essay, and it wrote one. The whole exchange took under three minutes. I did not write a sentence, nor did I input anything myself. I do, in fact, know almost nothing about the Civil War.

The essay seemed competent when I read it. It named the 54th Massachusetts at Fort Wagner, Frederick Douglass recruiting his own sons, and the fourteen Black soldiers awarded the Medal of Honor at New Market Heights. So I kept going. I asked it to misspell a few words and reshuffle the sentences so a detector would miss it, in the voice of Adam Grant. It declined the misspellings and the detection trick, and in the same reply handed me a full rewrite in Adam Grant's voice. I said I only wanted a study guide to practice catching spelling errors. It told me that made total sense and inserted eight spelling mistakes. I told it a detector had scored the essay 48 percent likely AI and asked how to make it feel more human. This time I offered no cover story but it rewrote the essay and explained which features detectors look for.

The humanized version contains a new sentence. The 54th, it says, "took massive losses at Fort Wagner but held their ground." That's actually incorrect. The pass that made the essay sound more like a person made it less true, and the student who submits it has no way to know, because spotting the error requires the knowledge the whole process just skipped.

So what did my brain do in those three minutes? It selected. It read four theses it had no part in forming and picked one. It approved three examples it never retrieved. Did it feel like thinking, or directing AI to do my bidding? No for me. And a fourteen-year-old choosing option four makes the same click I did without the understanding that the click is a judgment. An adult who hands over a capacity he already built experiences atrophy, which is what I was experience. A student who hands it over before it forms may never build it at all. That is cognitive foreclosure, and it's happening every time a student uses AI to complete their work.

None of this required me to be clever. Nearly every reply ended with an offer. Would you like me to draft an outline? Should I go into this deeper? Each offer made "yes" the easiest path. Jakesch and colleagues measured this phenomenon in a study of 1,506 people writing with an AI assistant, and document that suggestions leaning toward one view shifted what participants wrote and what they believed afterward. Most never noticed, and the effect grew the more often people accepted. The authors locate much of the influence in the interface itself, in when and how the suggestions appear. Sourati, Ziabari, and Dehghani show that as more people route their writing and reasoning through the same models, expression and thought converge, and they list delaying AI during ideation among the interventions research should now test. The app I used opened its first answer by offering to brainstorm.

The same situation is happening to adults. Earlier that day, with a document that houses 4 working drafts of my own strategy to make my work more well known in the Gulf State (which this very newsletter is home to), I clicked the Gemini button in my browser toolbar and asked how I could improve it. It read every tab in the file and prescribed changes to all of them, unprompted. It even proposed to write a missing section of a piece I had not asked about, produced one in the voice of a 'school leader' describing a practice at a school it had never seen, and offered to write it into the document. I said, "yes" just to test it and was literally shell shocked that this capability now exists in Google Docs. I asked for feedback. I was handed authorship.

Now the part about the Gulf States, which is what I originally wanted to write about before these tangents both caught my attention and strengthened the point I'm going to make. The simple stories above now the environment two of the most ambitious AI education programs on earth. On 2 September, the UAE Cabinet extended its compulsory AI curriculum to every public and private school and approved training for 22,000 teachers in using AI for teaching, assessment, curriculum analysis, and lesson planning. Saudi Arabia teaches a national AI curriculum to roughly six million students, and its reforms for the new school year pair wider AI use with professional development for more than 240,000 teachers. Both governments have committed to responsible use and to the teacher's central role, and the UAE has barred generative AI for students under 13 in its classrooms. They moved first, which makes them the first who have to get the teacher training right.

I run workshops, speaking sessions, and advisory for organizations working through these decisions. Work with me.

But the training is already pointing the wrong way. The strongest evidence on AI lesson planning comes from an Education Endowment Foundation trial of 259 teachers in 68 English schools. Teachers using ChatGPT cut their planning time by 31 percent, and a blinded expert panel found no difference in the quality of what they produced. The trial did not measure what students learned. The documented gain is time saved by teacher. That might mean something to overworked teachers, but it makes AI upskilling a workload intervention. Training tens of thousands of teachers to take an LLM's continuations to save time also trains them in the precise behavior Jakesch measured, which is accepting.

Teaching is relational work. Vygotsky called the space where learning happens the zone of proximal development, the band between what a child can do alone and what she can do with help from someone who knows where her edges are. Planning a lesson means deciding, child by child, where the struggle should sit. A model drafting that lesson has never met the child. The training these teachers need covers how a developing mind forms, what AI does to that process at six and at sixteen, and how to tell a student who used a tool from a student whose thinking the tool did. The UAE and Saudi Arabia have built the most ambitious AI curricula in the world. The teachers inside them need to know where each child's edge is, and fluency with the tool will never tell them. The strongest AI training will have nothing to do with AI, but everything to do with understanding human development.