AI companies are increasingly exerting active thought control over individuals and financial markets.
asserted
companies → exert → individuals
Protect their freedom of thought by investing in sovereign AI, diversifying AI supply chains, and maintaining independent judgement, urges Lewis Liu
I recently spoke on a panel in front of senior financial regulators, bankers, and lawyers in New York about an increasing risk of “systemic qualitative risk”, a term I coined to describe potentially dangerous herd behaviour in our financial system, caused by everyone using the same AI model, calibrated by and/or to the benefit of the frontier labs (OpenAI, Anthropic).
asserted
I → protect → labs
In my talk, I also discussed why I had suddenly deleted Claude, and my concerns about Anthropic’s motives.
asserted
I → discuss → motives
What scares me is what came next.
asserted
what → scar → me
I drafted a post and had ChatGPT edit it.
asserted
ChatGPT → draft → it
In the draft, I summarised my talk where I gave two key reasons why I deleted Claude.
asserted
I → summarise → Claude
One: a few months ago, I wrote a column about AI sovereignty, was somewhat critical of Anthropic, and when I took it to Claude to copyedit, it silently removed every criticism I had made of the company.
asserted
I → write → company
I then discovered that Anthropic’s system prompt and constitution, their overarching guidelines for the Claude chatbot, explicitly state that Anthropic’s interests come ahead of the user’s.
asserted
interests → discover → user
Two: Anthropic increasingly believes they are the only group of people worthy of controlling the machine god they are ushering into existence, and that everyone else must be stopped.
asserted
everyone → believe → existence
And it is now well reported that it is increasingly trying to influence houses of worship and religious leaders to that end; my wife, being Irish Catholic, encountered a flavour of this grassroots campaign by Anthropic at her church.
asserted
wife → report → church
Normally, ChatGPT might “gaslight” me when it thinks what I am saying falls outside its understanding of the world, something I have written about before.
uncertain
I → gaslight → world
But this time, it actively tried to persuade me NOT to go into Anthropic’s motives.
asserted
it → try → motives
This is for sure not implicit bias from training data, but an explicit attempt to nudge users’ thinking about the AI industry.
asserted
This → nudge → industry
I had expected this behaviour from Anthropic (given their attitudes about themselves), but not from OpenAI.
asserted
I → expect → OpenAI
What is scary for me is that this goes way beyond AI bias, where AI reflects the most common, often biased, point of view, something we have all discussed before.
asserted
we → go → view
Instead, this active, wilful attempts by AI companies to control what and how we think under the guise of “AI safety”.
asserted
we → control → safety
This is active manipulation: to the benefit of, or aligned with, the creators of the model.
asserted
This → align → model
Imagine a world in which investment banks, hedge funds, asset managers and regulators increasingly use ChatGPT, Claude, Copilot and thousands of applications built on top of those same underlying models to analyse markets and make investment decisions.
asserted
banks → imagine → decisions
If those systems share similar assumptions or preferences, supposedly independent professionals can begin converging on the same conclusions without realising where that consensus came from.
asserted
consensus → share → where
This sounds dangerously close to conspiracy theory until you look at Anthropic’s own research.
asserted
you → sound → research
In a recent experiment, Anthropic gave professional analysts an AI agent with a randomly assigned preference on a series of judgement calls.
asserted
Anthropic → give → calls
The analysts followed the agent’s preferred choice 86 per cent of the time.
asserted
analysts → follow → time
Afterwards, 77 per cent used the agent’s arguments to explain their own decisions, sometimes almost word-for-word, but yet only 15 per cent credited the AI as the source of those arguments.
asserted
cent → use → arguments
Even more strikingly, the analysts remained confident that the final decisions represented their own judgement.
asserted
decisions → remain → judgement
This is precisely the systemic risk I worry about.
asserted
I → worry → ?
Financial markets depend on supposedly independent actors making independent judgements.
asserted
actors → depend → judgements
If thousands of those actors are actually receiving “thought nudges” from a small number of underlying models, their decisions may no longer be genuinely independent at all.
uncertain
decisions → receive → models
Imagine, for example, if those models systematically framed the AI sector more positively, or consistently weakened arguments critical of frontier labs.
asserted
models → imagine → labs
No single recommendation needs to be dramatic.
asserted
recommendation → need → ?
A tiny nudge, repeated millions of times, can become a very large capital flow – say to the benefit of those same model providers.
asserted
nudge → repeat → providers
And it gets more worrying as AI becomes agentic.
asserted
AI → get → ?
Many of these decisions will increasingly happen without a human ever seeing the intermediate reasoning.
asserted
human → happen → reasoning
According to Openrouter, an LLM model aggregator, agentic API keys recently consumed roughly seven times as many tokens as human API keys.
uncertain
keys → accord → keys
The machines are increasingly talking to machines.
asserted
machines → talk → machines
What can we do to retain our freedom of thought?
asserted
we → do → thought
AI is too powerful to ignore, and too far out of the bag to put back in.
asserted
AI → ignore → bag
First, the answer isn’t just pure open source, though open source plays a role.
asserted
source → play → role
I’ve discussed the Western frontier models here, but Chinese open-source models have their own political nudges too (Taiwan, Tibet, etc.).
asserted
models → discuss → nudges
I think there are three things we need to do, and the timing couldn’t be more urgent:
Invest in sovereign AI, for both yourself and your company
At my start-up, we are seeing more and more requests from companies wanting private or independent deployments.
asserted
we → think → deployments
Talking to other founders, this seems to be a growing trend.
asserted
this → talk → founders
…and 15 more, not listed.
The Aporia
