Much to my amusement, the world seems to have discovered Effective Altruism.
asserted
world → seem → Altruism
As the debate over AI safety and AI regulation has heated up, the national press is finally paying attention to the odd cultural bubble within the tech industry and tech-related circles of San Francisco Bay Area culture that produced much of the thinking on the topic up until now.
asserted
that → heat → topic
For many years, Effective Altruism — or EA, as it’s known1 — was a cross between a Berkeley technohippie cult and a fun intellectual sideshow that offered nerds a pleasant distraction from whatever people in mainstream politics were yelling at each other about.
asserted
people → offer → other
Part of that involved thinking a lot about the future of artificial intelligence, and a lot of the tech people who would end up working at AI companies — especially Anthropic — were very influenced by those discussions.
asserted
who → involve → discussions
So now that AI is the most important thing in the world, people in the outside world are discovering this strange little corner of our culture for the first time, and it’s very amusing to watch.
asserted
it → discover → time
You have mainstream news outlets publishing things like “This well-meaning ideology fueling AI panic has a dark side”, “How Effective Altruism Took Over the World”, “The Dangerous Ideology Behind the AI Warnings”, and so on.2
asserted
Altruism → have → Warnings
As a result, you’re now seeing people actually paying attention to some of the more far-out ideas in the EA community, like this one by the blog Bentham’s Bulldog:
asserted
people → see → blog
The notion that insects, in the aggregate, matter more than human beings received a huge amount of scorn.
asserted
insects → matter → scorn
But in fact, thinking about insect welfare versus human welfare is an important mental exercise, because it demonstrates some of the fundamental flaws in utilitarianism.
asserted
it → think → utilitarianism
Those flaws were minor side-notes up until now, but in the age of AI they’re going to become increasingly uncomfortable and glaring.
asserted
they → go → AI
As an economist, I was inculcated with utilitarianism from day 1.
asserted
I → inculcate → day
The notion that people ought to get what they want underlies essentially all of modern economic thinking.
asserted
they → get → thinking
GDP, at its most simple, is a measure of the degree to which people in our society get what they want.3
asserted
they → get → what
In some sense, economists adopted utilitarianism as a sort of defensive measure — instead of going out on a limb and endorsing some politically contentious concept of virtue or justice or national greatness or whatever, they could just say “We just want to give people what they want.”
uncertain
they → adopt → what
But even though it felt like a bare-bones sort of moral vision, valuing humanity’s aggregate utility can have some pretty strong policy implications.
asserted
valuing → feel → implications
Elon Musk wouldn’t even notice if his bank account went up or down by a billion dollars.
asserted
account → notice → dollars
But to plenty of regular people, a hundred thousand dollars would be life-changing.
asserted
dollars → change → people
Elon wouldn’t even notice, and the 10,000 people would get a lot more of what they want in life — better health care, an education for their kids, nicer food, a bigger house, or whatever.
asserted
they → notice → kids
How far can you push that principle, though?
asserted
you → push → principle
The above example — redistributing just 0.1% of the wealth of the world’s richest man — seems clear enough.
asserted
example → redistribute → man
But you can just keep designing more extreme thought experiments, until you end up asking whether bugs’ lives matter more than humans’ lives, and everyone just starts laughing at you.
asserted
everyone → keep → you
I titled this post “The problem(s) with utilitarianism”, but that was a bit of clickbait — people have been pointing out the problems with utilitarianism for over two centuries.4
asserted
people → title → centuries.4
The principle of “the greatest good for the greatest number” doesn’t take a stand on who gets to be happy and who has to be sad.
asserted
who → take → stand
Would it be good to have a society where one very greedy person gets more and more of what he wants, but never comes any closer to being satisfied, while everyone else is kept on the edge of starvation?
asserted
everyone → have → starvation
What about a society where one person gets perpetually tortured but millions live in blissful utopia?
asserted
millions → torture → utopia
This is “The Ones Who Walk Away From Omelas.”
asserted
Who → walk → Omelas
It’s not hard to come up with mental examples that satisfy the formal principles of utility maximization but which obviously seem grossly unfair.
asserted
which → ’ → maximization
Another problem is that the set of people whose utility you’re maximizing isn’t fixed.
asserted
you → maximize → utility
Is it better to have a very large population living on the edge of abject poverty, or a small number of people living in lavish comfort?
asserted
it → have → comfort
According to strict utilitarianism, these are equally good outcomes, because they involve the same total amount of utility.
uncertain
they → accord → utility
But most people would probably prefer one or the other.
asserted
people → prefer → one
Some would say that it’s good to be fruitful and multiply — that even if everyone is poor, it’s better to be poor than to never have existed.
asserted
it → say → ?
Others will say that life on the edge of starvation isn’t worth living, and that it would be better to limit population to an amount that can be comfortably sustained.
asserted
that → say → amount
In general, utilitarianism has a problem dealing with death.
asserted
utilitarianism → have → death
If someone is dead, does that mean they have zero utility, because nothing matters one way or another to the dead?
asserted
nothing → mean → dead
Or does it mean they have negative infinity utility, because lots of living people will try to avoid death at any cost?
asserted
lots → mean → cost
This isn’t just an abstract math question — lots of moral questions hinge on how much you value life versus quality of life.
asserted
you → hinge → life
For example, the fundamental question of AI risk — what chance of extinction we should accept in exchange for a shot at utopia — depends crucially on how much we value survival.
asserted
we → accept → survival
Things get even more confusing when we weigh the utility of real people who are alive today against the utility of potential people who might be alive later on.
uncertain
who → get → people
For example, in his book Stubborn Attachments, Tyler Cowen urges us to put a lot of weight on the well-being of our descendants in the far future.
asserted
Cowen → urge → future
…and 74 more, not listed.
The Aporia
