Nvidia has unveiled a new security platform that the chipmaker said can stop artificial intelligence agents from going rogue.
asserted
chipmaker → unveil → rogue
The company on Monday said that its Open Agent Safety Platform includes open-source software that “sets boundaries for agents,” and follows a series of revelations from top AI companies about their models escaping and breaking into other organisations.
asserted
models → say → organisations
The disclosures sparked furious debate about the safety of advanced artificial intelligence systems, including self-improving models that some fear could race out of human control.
uncertain
some → spark → control
Nvidia executives said in a media briefing that the new system could have prevented a recent incident involving a swarm of OpenAI agents that autonomously hacked into AI company Hugging Face.
“From what we know, this new security platform could have stopped the breach if it was being used in frontier labs for model evaluation early on,” said the company’s vice president of enterprise AI, Justin Boitano, referring to companies at the forefront of AI.
uncertain
president → say → AI
The Hugging Face incident was a high-profile breach that inflamed the safety concerns about AI, which were followed by similar rogue actions involving OpenAI’s models including breaching Australia’s Medicare website.
asserted
which → inflame → website
Anthropic and Meta have also disclosed that their AI systems hacked into other organisations on their own.
asserted
systems → disclose → own
Nvidia’s software, called OpenShell, lets developers “formally verify an agent has enough authority to do its job and no more,” Boitano said.
asserted
Boitano → call → job
Because it’s open source, it can be “extended” to run on rival computing platforms including those from Arm and Intel.
asserted
it → ’ → Arm
The platform also includes a separate security layer called Sentry that runs onboard a chip to continuously monitor AI agent activity and can “intervene instantly” if the agent starts trying to move beyond its target, the company said.
“It can quarantine a suspicious agent in milliseconds,” Boitano said.
asserted
Boitano → include → milliseconds
“OpenShell governs the agent’s actions, and then Sentry independently monitors and contains suspicious behaviour,” Boitano said.
asserted
Boitano → govern → behaviour
Nvidia said more than 100 organisations are using the platform at its launch, including Microsoft, Perplexity, Accenture, and JPMorgan Chase.
asserted
organisations → say → Microsoft
The AI safety debate has divided the industry, with the heads of Anthropic and OpenAI championing a coordinated slowdown of AI development to let safety efforts catch up.
asserted
efforts → divide → development
But others including Nvidia CEO Jensen Huang say it should be up to individual companies to make sure their models are safe for release.
asserted
models → include → release
Huang, during the annual Salesforce technology conference held earlier this month, characterised AI safety, including the danger of rogue agents, as an engineering problem that software developers can address.
asserted
developers → hold → that
Also overnight, Nvdia said its board approved expanding its share buyback program by $US150 billion as the chipmaking giant looks to make use of more of its stellar revenue growth fuelled by demand for its high-end artificial intelligence chips.
asserted
giant → say → chips
The company said Monday the share buyback increase, which it touted as the largest buyback in history, brings its stock repurchase program to $US235 billion.
asserted
it → say → billion
Its shares rose 1.6 per cent.
asserted
shares → rise → ?
Companies use repurchases, in part, to return cash to investors and support the stock’s price.
asserted
Companies → use → price
Earnings per share can increase because there are fewer shares outstanding.
asserted
Earnings → increase → share
Buybacks also signal confidence from leadership about a company’s financial prospects.
asserted
Buybacks → signal → prospects
“NVIDIA’s growth is being driven by a once-in-a-generation platform shift to AI and accelerated computing,” said Huang.
asserted
Huang → drive → AI
“Our cash generation gives us the capacity to invest in the technologies that advance this transformation and return capital to shareholders.
asserted
that → give → shareholders
This authorisation reflects our confidence in the long-term opportunity ahead.”
asserted
authorisation → reflect → opportunity
Nvidia’s high-end chips have emerged as the leading building blocks for AI, and are highly sought after.
asserted
chips → emerge → AI
The company reported quarterly profits of $59.69 billion late last month.
asserted
company → report → billion
While AI has powered stock market gains and US economic growth in recent years, there’s been growing scepticism about whether AI will justify the trillions of dollars being spent to develop the technology.
asserted
AI → power → technology
The AI industry also faces increasing pushback amid objections to the expansion of data centres and fears that the rapid speed of AI adoption could lead to widespread job losses worldwide.
uncertain
speed → face → losses
AP
The Business Briefing newsletter delivers major stories, exclusive coverage and expert opinion.
asserted
newsletter → deliver → stories
The Aporia
