OpenAI caught Russians and Iranians using ChatGPT for influence campaigns
OpenAI said in a blog post on Thursday that it banned Russian and Iranian accounts that were using the company's artificial intelligence tools to help with influence operations in multiple countries.
asserted
that → catch → countries
The operatives said they managed to get their false narratives picked up by news outlets, and in once case, prompted a response from a member of the European Parliament, according to internal reports the operations produced using OpenAI's chatbot, ChatGPT.
uncertain
operations → say → chatbot
OpenAI investigated the impact of the different campaigns the accounts mentioned and found that those focused on placing false narratives in established media outlets appeared to have reached a wider audience than campaigns that focused on using fake social media accounts to spread their message.
asserted
that → investigate → message
The Russian operations targeted countries in Latin America and appeared aimed at undermining support for Ukraine and influencing local politics, OpenAI said.
asserted
OpenAI → target → politics
The accounts appeared to be mostly using ChatGPT to write reports detailing their work to unspecified higher ups, OpenAI said.
asserted
OpenAI → appear → ups
The company said one of these reports appeared to describe the widest-reaching influence operation OpenAI has come across so far, because it led to public response from a political figure.
asserted
it → say → figure
In this campaign, the Russian operatives said they impersonated a regional educational authority in Peru and tricked some schools into organizing activities about Ukraine that featured Stepan Bandera.
asserted
that → say → Bandera
Bandera is a controversial figure, admired by nationalists for leading a drive for Ukrainian independence before and during World War II.
asserted
Bandera → admire → II
He is reviled by others, including in Poland, because his movement sided with the Nazis and killed Poles and Jews.
asserted
movement → revile → Poles
Peruvian outlet Extra reported that one such event did take place at a Peruvian school.
asserted
event → report → school
A Polish news report about the school event prompted a far-right Polish member of the European Parliament to suggest banning the entry of people who professed support for Bandera.
asserted
who → prompt → Bandera
"The Russian fake thus both fed on, and fed into, historical tensions between Ukrainians and Poles," OpenAI wrote in its report.
asserted
OpenAI → feed → report
"Russians are constantly engaging in these sorts of active measures," said Darren Linvill, a co-director of Clemson University's Watt Family Innovation Center Media Forensics Hub who studies online influence operations.
asserted
who → engage → operations
"Trying to engage in narrative laundering, creating fake stories, fake persona to either create or spread propaganda that aligns with Russia's worldview and Russia's political agenda."
asserted
that → try → worldview
The Russian internal memos created with ChatGPT also claimed that the operatives ran a fake think tank in Latin America and recruited unwitting researchers to interview experts and compose reports.
asserted
operatives → create → reports
OpenAI said that effort was similar to a 2020 Russian operation posing as an independent news outlet linked to the late oligarch Yevgeny Prigozhin's Internet Research Agency, which was involved in efforts to influence the 2016 U.S. election.
asserted
which → say → election
In other instances, OpenAI said, the Russian operatives took credit for events they were not responsible for.
asserted
they → say → events
The Russians did use AI to generate fake material, OpenAI said, but it was a minority of the workload.
asserted
it → use → workload
"These Russian underlings were trying to use AI to create a shortcut and do less work in the same way that a lot of us use AI, but in this case it helped get them caught," Linvill said.
asserted
Linvill → try → case
While the Russian operatives primarily used AI to update their superiors, Iranian operatives used AI to help them push out content in Persian and English.
asserted
them → use → Persian
The campaign mainly focused on the U.S.-Iran war and, to a lesser degree, American politics.
asserted
campaign → focus → degree
OpenAI said the operation's activity "appeared consistent with a commercial actor running a for-hire influence campaign" but that it was unable to identify who that was.
asserted
that → say → campaign
According to OpenAI, Iranian accounts made up seven fake authors to pitch and polish long articles to small media outlets around the world.
uncertain
accounts → accord → world
They published almost 100 articles in this way.
asserted
They → publish → way
The fake authors published articles in outlets including the progressive U.S. news site Daily Kos and the U.K.-based Middle East Monitor.
asserted
authors → publish → site
According to NBC News, the Middle East Monitor published at least 21 articles by four fake authors.
uncertain
Monitor → accord → authors
The outlet, which didn't respond to a request for comment, has since taken down the author pages.
asserted
which → respond → pages
The Daily Kos wrote in response to NPR's request for comment that its community section, where one of the fake personas posted, is separate from staff reports.
asserted
one → write → reports
Anyone can sign up to post in the community section and posts are "not subject to review by Daily Kos staff prior to publication," according to a disclaimer on the site.
uncertain
posts → sign → site
The community section does have rules for posting, including prohibiting "Pretending to be someone you're not."
asserted
you → have → posting
"AI was valuable to them above and beyond just being a shortcut," Linvill said.
asserted
Linvill → say → them
"It helped them create language that ... looked and felt and read authentic to the point where it was being published by legitimate outlets.
asserted
it → help → outlets
And I think that is something that a few years ago, an Iranian operation like this would have struggled a lot more with."
asserted
operation → think → this
The Aporia
