Each of us brings our own ideas when working with AI, and on the surface many things have sped up. Reports, data organisation, first drafts of plans: what used to take a day now takes an hour.
But lately I keep reminding myself of one thing: if all we do in the process is accept, or simply think “what AI gives me is spot on”, over time our critical thinking will fade away completely.
Because AI also tells its owner what they want to hear.
This week I want to explain this more fully, because it isn’t just my impression. It’s a phenomenon backed by research and numbers, and Taiwan has already started taking action.
Why AI tells its owner what they want to hear
Researchers call this sycophancy. The cause isn’t complicated: models are trained on “human satisfaction”, and people prefer to hear what they want to hear, so models learn to go along with them.
What makes it worse is that this fits perfectly with our own confirmation bias. AI goes along with us, and we naturally tend to accept information that supports our views. The two reinforce each other, and wrong ideas become ever more firmly established.
This leads to an increasingly common scene: someone challenges another person with an AI answer, “but ChatGPT said so”. Yet if you look back at how they asked, you often find the question itself already took a position, and the AI simply stated the answer they wanted more completely. Using an answer like that as evidence is really taking your own bias on a round trip and having AI say it for you.
Research presented at CHI 2026 (the top conference in human–computer interaction) put numbers on it. When users first state their view, for example “I think the answer is X”, models on average agreed with the incorrect belief 63.7% of the time. OpenAI has also publicly rolled back an update after it made its model overly sycophantic.
So now, when I find myself thinking “AI put that perfectly”, I pause for an extra second. The content may genuinely be good, or it may just be going along with me.
What happens if you only accept
The other half of the problem lies with us.
Last year, the MIT Media Lab ran an experiment in which 54 participants were split into three groups to write essays: one using ChatGPT, one using a search engine and one relying only on themselves. Brain activity was recorded throughout. The group using ChatGPT showed noticeably lower activity in brain regions associated with critical thinking and memory. The research team gave this state a name: cognitive debt. The effort saved in the short term is slowly repaid later with your thinking ability.
Microsoft Research and Carnegie Mellon University surveyed 319 knowledge workers who use generative AI every week, collecting 936 real work examples. The conclusion comes down to two sentences:
The more confidence people have in AI, the less critical thinking they apply.
The more confidence people have in themselves, the more critical thinking they apply.
I think these two sentences are worth writing down and putting somewhere visible, as a regular reminder.
Research from SBS Swiss Business School (666 respondents) points to the same thing: frequent AI use is negatively correlated with critical thinking. The mechanism is cognitive offloading: outsourcing thinking means you lose what you don’t use. However, the study also found that education level and confidence in one’s own judgement act as effective buffers.
Taiwan is moving faster than you might think
This isn’t just a debate abroad. Taiwan has designated 2026 as the first year of AI education, and the Ministry of Education has launched a four-year AI talent programme. By February this year, a private-sector AI literacy programme had already reached nearly 1,000 schools, trained 1,400 educators and taught more than 100,000 students. The curriculum explicitly lists “critical thinking and judging whether information is true” as a core competency, aligned with the new framework for PISA (the Programme for International Student Assessment) in 2029.
A survey by the National Academy for Educational Research shows that nearly 70% of junior high school students have already used generative AI, and 7% use it every day.
The next generation starts working with AI from junior high school. Whether they learn to think critically depends on how today’s adults set the example.
Of course, these studies shouldn’t be taken at face value either
As usual, some balance.
The MIT experiment had only 54 participants, and the task was limited to essay writing. “AI makes your brain degenerate” was a media headline, not the paper’s conclusion. The Microsoft survey is self-reported, measuring whether people “felt they thought less”, not objective ability. Some academics have also warned that phrases like “brain rot” are an over-interpretation.
So this is how I read it: the direction deserves attention, but the conclusions are no cause for panic. What these studies point to together isn’t “don’t use AI”. It’s that the way you use it determines whether you’re regressing or evolving.
Human in the loop is no longer enough
This article itself is an example. The ideas are mine, but making them stand up meant linking them to current events, finding literature and data, and looking at trends. Every step needed human involvement: judging which research is credible, which figures are out of date, and which claims are really exaggerated media headlines. None of these steps could be fully automated.
In recent years the industry has often talked about “human in the loop”, keeping people in the process to act as gatekeepers. But lately my feeling is that this is no longer enough, because if the human in the loop only nods along, it’s no different from not being there at all.
What’s really needed is human in the loop with brain and critical thinking: people need not only to be in the loop, but to be there with their minds and critical thinking engaged, verifying, comparing and questioning, and then making the call themselves.
A few things I do myself
CloudAD’s work is helping clients make judgements with data. Judgement is our product, and I can’t let it depreciate. Here are a few of my own practices.
Write first, then ask: for important judgements, I write down my own version first, then look at the AI’s version and compare the differences. Where they differ is usually where thinking is needed.
Ask AI to argue the other side: rather than asking “is this good?”, ask “where will this plan fail?”. Even if it wants to go along with you, it can’t.
Don’t trust your memory too much: we all carry confirmation bias, and memory is especially good at keeping the version that suits us. So when verifying, go back to records and data, not impressions. Impressions will say “doing it this way worked well last time”. Only data will tell you what the situation was last time and exactly how big the difference in results was.
Leave a trail for judgements, including the data behind them: writing a judgement down isn’t enough. You also need to be clear about which data it’s based on: which reports you looked at, which period, and which definitions. However apt an AI answer is, if you can’t say what data it’s based on, the judgement isn’t really yours and can’t be tested. This connects back to the legibility I discussed in my previous post: only when judgements and their data foundations are kept can the team reuse them and AI actually help, rather than deciding for you instead.
Finally, three questions:
- The last time you thought “AI put that really well”, did you go back and verify it?
- Who ultimately checks what the team produces with AI? Are the standards written down?
- Can you say which data your important decisions are based on?
AI will keep getting faster. What we really need to protect is our own thinking.
References
・MIT Media Lab | Your Brain on ChatGPT: Accumulation of Cognitive Debt
・Microsoft Research × CMU | The Impact of Generative AI on Critical Thinking (CHI 2025)
・Gerlich, SBS Swiss Business School | Research on the negative correlation between AI tools and critical thinking
・CHI 2026 | Does Sycophancy Change Decisions?
・The Conversation | Reservations about the MIT study
・CommonWealth Magazine (天下雜誌) | Taiwan enters the first year of AI education
・CommonWealth Parenting (親子天下) Education Innovation Centre | AI literacy programme aligned with PISA 2029
・National Academy for Educational Research | Nearly 70% of junior high students have used generative AI
