Governing the unmanageable: what happens when AI builds a population of thinkers

AI, cognition, and the governance crisis nobody is building for

Bola Ogunlana  |  Cockpit to Cloud  |  May 2025  |  12 min read

Somewhere in the next decade, governments are going to discover they’ve been managing the wrong population. The one they were built for followed instructions. The one arriving will not.

AI is changing cognition at scale. Peer-reviewed research confirms it. The argument isn’t settled on whether it’s making us smarter or dumber (both are happening, depending on who you ask and what they’re doing), but the direction of the change is clear enough to worry about. Specifically, to worry about what happens to the governance structures, management hierarchies, and political institutions that were quietly built on the assumption that most people would stay cognitively compliant.

That assumption has a 140-year history. And it’s about to be stress-tested.

The world Taylor built

Frederick Taylor stood in a Pennsylvania steel mill in the 1880s and watched workers think. He decided that was the problem.

His solution, scientific management, stripped decision-making out of the hands of workers and concentrated it in managers. Jobs were broken into the smallest possible components. Workers executed. Managers planned. Thinking was a supervisory function.

That model didn’t stay in the steel mill. It spread into civil services, hospitals, schools, armies, and parliaments. The whole architecture of modern governance, top-down authority, procedural compliance, hierarchical accountability, was designed for a population that mostly followed instructions. Industrial age jobs were typically low-discretion, required little decision-making, and were broken into simple tasks which required very little thinking or judgement. That was the explicit design brief.

The Information Age loosened this slightly. Knowledge workers needed some discretion. But the structural incentives still rewarded staying in lane. Most people, doing most jobs, in most organisations, operated within a narrow band of permitted independent thought.

Governance worked because the governed mostly cooperated with that arrangement. They were busy, tired, and had been trained since childhood to defer to expertise and authority. It was easier.

AI is about to make it harder.

The split nobody talks about clearly

Here’s the honest picture from the research, stripped of optimism and alarm in equal measure.

Michael Gerlich at SBS Swiss Business School published a 2025 peer-reviewed study of 666 participants across diverse age groups and educational backgrounds. He found a significant negative correlation between frequent AI tool use and critical thinking ability. The more people leaned on AI, the less capable they became at independent analysis. Younger participants were the most dependent and scored lowest on critical thinking assessments.

So AI is, in some cases, making people worse at thinking. That’s not a moral panic, it’s measured data.

But then Microsoft Research published findings from CHI 2025, surveying 319 knowledge workers across 936 real-world AI use cases. Their finding: for high-stakes tasks requiring accuracy, knowledge workers expend more critical thinking effort when using AI than they would performing the same tasks without it. The AI demands interrogation. You have to verify, challenge, and synthesise outputs that you can’t simply trust. That cognitive load, applied consistently, builds rather than erodes analytical capacity.

Both studies are correct. And that’s the thing worth sitting with.

AI isn’t producing one cognitive outcome. It’s producing 2, simultaneously, in the same population, depending on the person, the task, and the stakes. One group is offloading thinking to AI and their analytical muscles are weakening from disuse. The other group is being forced to engage more rigorously than they ever did before, because AI produces confident, fluent outputs that are frequently wrong in subtle ways, and catching those errors requires genuine expertise.

The population is splitting. Call them the offloaders and the interrogators.

Why this matters for governance, specifically

Every governance system, democratic or otherwise, rests on a few working assumptions about the people being governed. The 2 most important ones are that most citizens are too busy or disengaged to scrutinise decisions closely, and that information reaches them through controlled intermediaries: official channels, mainstream media, accredited expertise.

Both assumptions are weakening simultaneously, and AI is the accelerant.

The “interrogator” population, growing steadily as high-stakes AI collaboration becomes standard, will not accept opaque decisions from public institutions. They deal with opaque AI outputs every day and have developed the habit of asking: where did this come from, what assumptions are baked in, and who is accountable if it’s wrong? That habit doesn’t stay at work. It comes home and it comes to the ballot box.

Researchers at LMU Munich identified what they call the “double delegation problem” in a May 2025 working paper. When AI systems handle parts of government decision-making, and politicians defer to those systems, the accountability chain breaks. Who is ultimately responsible when an algorithmic decision shapes a political outcome? Currently, nobody has a good answer. A critically thinking citizenry will not accept the silence.

“AI now functions as a critical cognitive layer, requiring strategic governance to protect human reasoning and judgement. Cognitive offloading and automation bias threaten the intellectual stamina required for national competitiveness and democratic stability.”

— World Economic Forum, March 2026

The WEF called this a strategic governance challenge in March 2026. They’re right, but they undersell how structurally disruptive it actually is. A critical citizenry doesn’t just want better information. They want genuine participation. And the tools to deliver that now exist.

The deliberation infrastructure is already being built

Taiwan has been running vTaiwan since 2015. When Uber launched in Taiwan and the regulatory conflict broke out, the government used vTaiwan’s AI-assisted deliberation platform to aggregate public input and reach a workable framework in months. A dispute that took years of lobbying and counter-lobbying in most Western countries was processed through genuine collective reasoning, with the AI synthesising thousands of individual contributions into structured decision-ready maps.

California launched “Engaged California” in February 2025, using similar AI synthesis tools to handle post-wildfire rebuilding policy. The technology did something that was previously impossible at scale: it preserved minority positions that would otherwise get swamped by majority preferences, giving smaller constituencies a genuine voice in the synthesis.

The Carnegie Endowment, writing in May 2026, put it this way: platforms like Polis, Talk to the City, and vTaiwan can aggregate thousands of individual contributions into structured thematic maps while preserving minority positions that would otherwise be submerged. The complexity of topics like post-disaster rebuilding, they noted, requires qualitative synthesis at a scale that only AI can manage.

So the infrastructure for a genuinely thinking population to actually govern exists. The question is whether existing political institutions will adopt it before the pressure from below forces their hand, or whether the pressure arrives faster than the institutions can absorb.

Which brings in the counterforce that complicates the whole picture.

Critical thinking requires honest information to work on

A thinking population is only as effective as the information environment it operates in. And AI is degrading that environment at the same time as it’s improving some people’s cognitive capacity.

If generative AI can flood media channels, social platforms, and personal correspondence with synthetic content that is indistinguishable from genuine reporting, the interrogators don’t necessarily win. They just have harder interrogations to perform against more sophisticated disinformation. And the offloaders, who’ve surrendered the habit of verification, become easy targets for whoever constructs the most convincing narrative.

This is the sharpest tension in the whole picture. AI producing thinkers while simultaneously producing the raw material to mislead those thinkers. It’s not a contradiction that resolves cleanly.

The techno-feudalism problem

There’s a darker argument sitting underneath all of this, and it’s the one most commentary skips because it’s uncomfortable.

The entire framing of “a thinking population” assumes that thinking is genuinely free. Political economist Yanis Varoufakis argues it won’t be. His “techno-feudalism” analysis, developed across his 2024 book and subsequent academic work, describes a system where digital monopolies function like medieval lords: controlling the terrain on which commerce, communication, and deliberation happen, and extracting rent from everyone who needs to use that terrain.

Amazon doesn’t make anything, in Varoufakis’s framing. It controls the digital system and charges “cloud rent” on all commerce in its domain. Google doesn’t just run a search engine. It controls what gets found. Meta doesn’t just connect people. It determines, algorithmically, what ideas spread.

A population of thinkers doing their thinking inside platforms owned by a handful of companies is still a managed population. The lords are just less visible than Frederick Taylor was.

A March 2025 academic paper put the structural point plainly: unlike past technological advancements, AGI is both a worker and an owner, producing economic value while concentrating power in those who control its infrastructure. Left unchecked, this shift risks entrenching techno-feudalism where intelligence itself becomes the most exclusive form of capital.

So the governance question isn’t just how governments manage thinking populations. It’s whether governments still have the sovereignty to manage anything meaningful, given how much decision-making authority has already quietly transferred to private platform infrastructure.

What this demands of management

This isn’t only a political problem. It’s a management problem, starting now.

Knowledge work is already shifting from material production to critical integration. The physical outputs, text, images, analysis, code, are increasingly handled by AI. What remains is the judgment about whether those outputs are right, and what to do with them.

You can’t manage that transition with industrial-era tools. Annual performance reviews, rigid job descriptions, hierarchical approval chains: none of these were designed for a workforce whose primary output is judgment under uncertainty. And the skills required to exercise that judgment, the ones that distinguish the interrogators from the offloaders, develop through practice, not compliance.

The AEI’s 2025 de-skilling economy report made a point that deserves wider attention: unemployment rates for workers with liberal arts degrees are now about half that of computer science graduates, based on 2023 Federal Reserve Bank of New York data. The market is already sending a signal. The people trained to read carefully, argue coherently, and question assumptions are more employable right now than the people trained to write code. Because the code is being automated, and the questioning isn’t.

Management structures built for compliance will struggle to retain and deploy people whose value is precisely their capacity to push back.

The path forward, stated without false confidence

The WEF’s March 2026 policy agenda identifies 4 interventions worth taking seriously: requiring AI systems to include features that promote active thinking rather than passive acceptance (structured evidence pathways, built-in counter-arguments for high-stakes tasks); national AI literacy frameworks that teach citizens to interrogate AI systems rather than avoid them; governance of high-influence AI platforms as cognitive infrastructure requiring transparency audits; and preserving meaningful human accountability in healthcare, justice, finance, and public administration.

These are sensible. The UK House of Lords has been pushing versions of them. Taiwan has operationalised several. The EU AI Act gestures toward the infrastructure governance piece.

Most governments have produced reports and commissioned more reports.

The gap between what’s technically achievable and what political will can deliver is the honest uncertainty in this picture. And it’s a large gap. The institutions most in need of reform are the ones that would need to reform themselves, and they were built, structurally, to resist exactly that kind of pressure.

The question left standing

There’s something underneath all of this that the policy papers don’t quite reach.

Industrial populations elected leaders who projected authority. Information-age populations elected leaders who projected competence, or at least the appearance of it. A genuinely critically thinking population, the interrogators, will likely select for something different: leaders who can sit with uncertainty in public, acknowledge what they don’t know, and reason transparently through hard decisions without retreating to scripted confidence.

That would be a real change. Maybe the deepest one.

Whether the political systems we have can produce those leaders, and whether the thinking population can hold together long enough to select them before the offloaders get captured by whoever builds the most persuasive AI-generated narrative, is the question that doesn’t have a research paper answer yet.