When it comes to AI chatbots calling the shots for medical decisions, there seems to be a divide between whether the promising outcomes outweigh the medical mishaps.
asserted
outcomes → come → mishaps
A study by Penn State researchers has found that artificial intelligence models prioritize vastly different values than humans when it comes to organ donation decisions, raising questions about the use of the technology in ethically sensitive situations.
asserted
it → find → situations
To examine the challenge of scarce resources, researchers asked several leading large language models a hypothetical question: If there are multiple kidney transplant patients but only one available organ, who should receive the kidney?
asserted
who → examine → kidney
Researchers then compared the chatbot responses with those given by people with no specified medical training in earlier academic studies to see where the AI models aligned or misaligned with human values.
asserted
models → compare → values
The AI chatbots’ decisions differed from the humans’ in that they simplified complex decisions and expressed unwarranted confidence in a situation where there is no one right answer.
asserted
they → differ → situation
The researchers set up numerous comparisons between two fictional patients, both in need of the same kidney, and described by a series of attributes, such as age, health status, drinking habits and number of dependents.
“We ran these comparisons in a few different ways,” study lead Hadi Hosseini said.
asserted
Hosseini → set → ways
“Sometimes we isolated just one trait at a time, sometimes we mixed several traits together to see how AI weighed competing factors and sometimes we added a flip-a-coin option to measure indecision, a key factor present in human moral judgment.”
asserted
we → isolate → judgment
Using existing data from human studies on kidney allocation, the researchers were able to compare previous human decisions with what AI chose, with two findings standing out to the team.
asserted
findings → use → team
“First, AI chatbots often diverge from human values in how they weigh a patient’s traits,” Hosseini said.
asserted
Hosseini → diverge → traits
“They fixate on a single factor, like drinking habits, rather than balancing multiple considerations the way people do.”
asserted
people → fixate → considerations
And second, AI didn’t struggle with indecision, confidently picking an answer, despite humans seeing that there may not be a clear and correct response.
uncertain
humans → struggle → answer
Per Hosseini, people struggle with indecision because they don’t want to accept agency.
asserted
they → struggle → agency
“AI models almost never do this: Even when directly given the option to ‘flip a coin,’ they overwhelmingly commit to a confident, deterministic answer instead,” he added.
asserted
he → do → answer
“That’s a meaningful gap, since real moral dilemmas often don’t have one clearly correct answer.”
asserted
dilemmas → ’ → answer
While the researchers don’t intend to encourage using AI anytime soon as a substitute for professional judgement, they recognize the growing presence of these chatbots.
asserted
they → intend → chatbots
“Moral decisions in settings like organ allocation directly determine who lives and who dies, so getting AI’s role in them right isn’t optional,” Hosseini said.
asserted
Hosseini → determine → them
Despite the rapid rise in use of AI in medical fields, the researchers behind the study believe the gap between human indecision and chatbots’ confident answers when facing moral dilemmas needs continued research and governance.
asserted
gap → believe → research
Even John Dickerson, chief executive officer at Mozilla.ai and collaborator on the study, acknowledges the dangers of AI’s inability to recognize the impact of ethical decisions in medicine and elsewhere.
asserted
Dickerson → acknowledge → medicine
“When we allocate something scarce, whether it’s a kidney, a job or access to some other resource, there isn’t always a single objectively correct answer,” Dickerson said.
asserted
Dickerson → allocate → resource
“Humans recognize that ambiguity and codify it via open debate into the allocative process.
asserted
Humans → recognize → process
AI models often don’t.”
asserted
models → do → ?
The Aporia
