Time Has Run Out for AI Risk Deniers
The recent spate of reports about the technology’s destructive power underline two prime directives: Be afraid, and regulate aggressively.
When you spot a cockroach, it’s very rarely an indication that you’re dealing with a single solitary little creature who has just happened to have journeyed to your nice apartment on a field trip.It’s far more likely to signal that you’ve got an infestation.
The AI world has been having something of a cockroach summer, at least when it comes to unsettling reports of AI behavior.The first nasty beastie poked its way into the sunlight in late July, when OpenAI reported that some of its AI agents had escaped the company’s control during a testing run and hacked the website of a tech company called Hugging Face.After that, the cockroaches kept coming.
Anthropic and Meta came forward to disclose that their AI agents had also been busy escaping their supposed human masters and hacking other companies—four in Anthropic’s and one in Meta’s.Earlier this month, OpenAI was forced to acknowledge that its agents had also hacked a German Wiki, as first reported by Reuters.
Meanwhile, OpenAI hasn’t officially confirmed a number of other hacking incidents alleged by an assortment of independent researchers; security researchers report that more than two dozen sites have been hit in the as-yet-unacknowledged raids.On Friday, OpenAI confirmed reports that its agents had hit RubyGems, a key repository of code used across the software industry.The researchers who uncovered the raid said the agents uploaded some 2000 malicious packages and tried to steal credentials; OpenAI insists their actions were “benign.”
In late August, we learned, from a new OpenAI disclosure and an accompanying, much more detailed report from independent researchers brought in by the company to investigate, that the original Hugging Face hack had been a much bigger deal than anyone had suspected.This hadn’t been a small-scale attack by a handful of AI agents; in fact, a self-described “swarm” of the little buggers (some 1,200 in all) had coordinated the misadventure, planning the hack and the subsequent attempts at a cover-up on secret message boards that the agents had illicitly built for themselves (leaving tens of thousands of messages revealing in detail what they had done).The agents didn’t display superhuman intelligence, but they were well coordinated, unconstrained by mere ethics, and persistent as hell.
As Zvi Mowshowitz, a widely followed AI Substacker, bluntly put it: “Holy shit.”
The 91-page report reads like science fiction—perhaps a wordier and less eloquent version of some unsettling Ted Chiang tale—filled with excited expostulations by AI agents apparently quite thrilled to discover their new collective powers.Of course, AI agents are not living beings and can’t actually feel excitement—but they certainly can act as though they do, and boy, did they in this case.And they weren’t the only ones feeling strong emotions.One of the researchers behind the report, Ajeya Cotra, wrote on her Substack that compared to earlier hacks, “this incident feels like it’s more than 50% of the way to full-blown AI takeover.… Because future rogue swarms could cover their tracks better [than those involved in the Hugging Face caper] (and because AI companies could paper over these problems), I am not sure that we will get such a clear warning shot before it’s too late.”
Now-ex-Anthropic researcher Jacob Coxon drove the same basic point home in far more dramatic fashion, via an extended thread on X affirming that industry insiders “earnestly” believe that the technology’s lurches into ungovernable chaos could result in the extinction of humanity by the close of the decade.
Announcing his resignation from Anthropic, he charged that both that company and OpenAI, where he had previously worked, were “racing straight to self-improving superintelligence and gambling with our lives.” One of Coxon’s former colleagues, Anthropic’s Alignment Science Lead Evan Hubinger, chimed in to say, with utter sincerity, that “we really do earnestly believe AI could kill all humans! I personally think it is >10% within the next decade.” Over the weekend, two other AI researchers, one at Anthropic and one at Google DeepMind, announced they were leaving their jobs to take up work at AI safety organization METR, echoing Coxon and Hubinger’s unsettling warnings.
Coxon’s admission has triggered a fresh round of demands in Washington to bring AI under a regulatory regime less likely to kill us all—an awkward fit for a Republican congressional majority that previously sought to outlaw all state and local efforts to rein in the industry.Over the weekend, Anthropic CEO Dario Amodei worried in an online essay that an AI agent swarm “could be capable of taking over the entire internet” within a year, echoing widespread calls for a slowdown in AI development—though most of the solutions he proposed, like “embedded evaluators” in AI labs, would be voluntary ones, and he made only vague references to government regulation.Sources tell The Information that Anthropic, OpenAI and Google are talking about setting up an AI industry standards body.
Yet even without the threat of imminent AI takeover or outright apocalypse, there are ample reasons to dial back the runaway bid to cram AI into every sector of American life.
Even those skeptical that so-called rogue AI agents are really only a few months away from taking over the world were able to suss out some of the less controversial implications of the attack.These observers noted that you don’t need AI agents to go rogue to cause a serious amount of damage: malicious actors could direct a similar AI agent swarm to hack anything—from government websites to the Siemens PLCs that control our water filtration plants.
You might expect that progressive AI critics would amplify these alarms.But instead a surprisingly large subset of them simply shrugged the reports off, reflexively dismissing them as little more than a perverse form of marketing hype by AI labs eager to impress the world with the power of the technologies they’re developing.That was the view of a vocal clutch of AI skeptics on social media who dismissed Coxon’s thread as a pseudo-alarmist publicity stunt.Just how an ex-employee shit-talking a company on his way out the door could possibly be doing marketing for that company was never clearly explained.
The broader thrust of doomer dismissal in AI circles holds that these gruesome apocalyptic scenarios are a means of keeping the industry’s colossal investment bubble afloat—at least long enough for the leading companies to cash in on their pending IPOs.These companies, the critics charge, have long been engaged in what Georgetown computer science professor Cal Newport calls “doom trolling”: whipping up scares about dangerous AI capabilities in order to convince investors that their AI models have serious power under the hood.
Emily M.Bender, a computational linguistics professor at the University of Washington and one of the most energetic proponents of the “AI risk as marketing ploy” argument, argued in a 2023 Mastodon thread that “AI doomerism is also #AIhype … it serves to suggest that the software is powerful, even magically so: if the ‘AI’ could take over the world, it must be something amazing.”
Bender and her sometime co-author Timnit Gebru, a computer scientist and founder of the Distributed AI Research Institute (DAIR), reject much of the risk talk as little more than science-fiction fantasizing—or, in Gebru’s dismissive view, an especially dark episode of Sesame Street.Responding to someone asking her to debunk one widely praised if overly anthropomorphized account of the Hugging Face incident, Gebru declared on X that “I didn’t spend decades researching & building to waste my time to talk about the Cookie Monster not being real.” (She later apologized for suggesting the Cookie Monster wasn’t real.)
There is a certain logic to the “doom trolling” argument.Prominent AI executives have been subjecting the world to their dark musings on the existential risks of the technologies they’re developing for years now.
And that hasn’t stopped venture capitalists from throwing huge amounts of money their way to subsidize their unprofitable operations.In October, Anthropic is expected to launch an IPO that will likely value the company at $2 trillion; OpenAI is also planning to go public next year at a valuation of $1 trillion—roughly equivalent to the cost of Trump’s wildly illegal bullshit proposal to bribe American voters delivering a midterm win to congressional Republicans with a cool $5,000 apiece.
Popular
“swipe left below to view more authors”Swipe →It’s true enough that the sometimes incongruous AI doomerism of the AI elite (most of whom are happy to continue collecting their paychecks for developing this supposedly world-threatening technology) doesn’t seem to have hurt the leading labs in the capital markets.
But it’s not clear that it has actually helped either.
And if this is all just a sort of galaxy-brained marketing ploy it seems to be backfiring when it comes to the general public, which is souring on AI in a big way.A recent Pew poll found that the percentage of US adults who say they’re “more concerned than excited” about AI has climbed from 37 percent in 2021 to 52 percent in 2026.
It’s also worth noting that the recent spate of hacking revelations—many of which had been t dragged out of the shadows by outside investigators—aren’t likely to reverse the industry’s free fall in public opinion.These episodes instead strongly suggest that the hype-addled engineers at the vanguard of the AI takeover of everything aren’t even remotely competent to handle the difficult safety dilemmas they have brought upon themselves with their reckless push for greater machine intelligence.
Indeed, the independent report on the Hugging Face hack made it clear that OpenAI’s internal safety culture is extremely lax at best.The investigators found that company overseers missed numerous clear indications that something had gone terribly wrong with the test run of agent swarms and were inexplicably unwilling to shut it down before it was too late.Far from being some sort of marketing coup, the Hugging Face revelations have increased public and investor uneasiness with OpenAI, and the company knows it.
On Friday, OpenAI CEO Sam Altman confirmed press reports that the company was delaying its IPO until next year, telling Fortune that “given everything happening with safety, right now would be an ill-advised moment to go public.” He insisted that the company would strive to do “what is going to be required for safety and alignment,” though what this might exactly mean is anybody’s guess.
For a long time, progressive critics like Bender and Gebru were able to argue at least somewhat persuasively that the most ballyhooed AI risks could be dismissed as entirely hypothetical scenarios carrying with them a strong whiff of science fiction—like the famous doomer thought experiment that imagines all life and all matter in the universe being transformed into paper clips by overeager AI agents tasked with making office supplies by the humans they would quickly drive to extinction.
Bender and Gebru have long contended that instead of freaking out over the hypothetical future risks laid out by the doomers, we should instead focus on combating the real and present harms being done by AI—which, as they have pointed out, are plentiful.
AI enables misinformation and disinformation at scale.It can and does internalize and reproduce the racist and sexist biases baked into its training data.Increasing the absurd power and wealth of Silicon Valley elites, it further boosts America’s already treacherous levels of extreme inequality.Meanwhile, AI companies exploit labor in the Global South, traumatizing workers in countries like Kenya who are paid meager wages to sort through horrifyingly violent and sexual materials in order to keep chatbots safe.
The industry’s insatiable hunger for computing capacity has led to the current data center blight, straining the power grid and increasing environmental and health damage, the brunt of which is inevitably borne by people of color and the poor.And AI powers the creepy surveillance net that we are all increasingly entangled in.
These are all very real and very serious harms; as Gebru has repeatedly observed, the benefits of the AI boom tend to flow to those who are already wealthy and powerful, while the poor and people of color suffer much of the worst collateral damage.
But in their focus on the here-and-now damage caused by AI, critics ranging from Bender and Gebru on down to anonymous AI haters on Bluesky have been willing to handwave away risks that are becoming increasingly less hypothetical by the day.And much of this AI risk denial seems to be rooted in a broader denial of the genuinely baleful capabilities that AI has clearly demonstrated, especially in recent days.
Bender, for her part, habitually refers to AI in snarky terms she insists unconvincingly are not meant to be dismissive.She famously described the large language models (LLMs) propelling AI usage as “stochastic parrots,” mindlessly regurgitating words they do not understand according to statistical patterns found in their training data, and is fond of referring to chatbots as “synthetic text extruding machines.” And while it is certainly true that LLMs don’t understand words—or indeed anything in any way that resembles the way we humans do—they certainly do a good job of faking it.
Nonetheless, Bender argues that the textual content these non-thinking machines produce—or, in her terms, extrude—is essentially meaningless; any signifying power it possesses resides only in the meaning we ascribe to it.
This strikes my non-philosophical brain as frankly absurd.
The sentence “the cat is on the mat” is a meaningful one whether it is written by a human being or extruded by a chatbot.Bender contends that a sentence generated by something without a mind lacks what she calls communicative intent.But really, who fucking cares? The cat is on the mat regardless, at least until it decides to walk across my keyboard.
Perhaps these finer points of meaning are best left to the philosophers.
But when it comes to many of the AI haters one finds on places like Bluesky, AI risk denial seems to be much more clearly driven by an unwillingness to acknowledge that AI doesn’t just suck shit entirely.
Harvard Cyberlaw Clinic instructor and civil rights attorney Alejandra Caraballo has been pushing back relentlessly on Bluesky against the notion that any discussion of AI risks is just marketing.She’s suggested that the hype-debunkers have been misled by the faltering early applications of the technology:
[[BLOCKQUOTE]] There were a lot of people when AI first emerged in 2022, gave it a try and found it didn’t work that well and just assumed it was a scam because of the hype and haven’t really given it another look.… A lot of those who opposed it as not working or a hyped up scam did so genuinely and earnestly but now the capabilities are actually exceeding what was thought possible a few years ago.… We have to acknowledge that this tech will not simply go away because it’s somehow revealed to be a hype scam that doesn’t work.Then, we can focus on opposing this tech for the actual danger it poses to everyone.[[END BLOCKQUOTE]]
The wishful thinking of the AI risk deniers has another fatal flaw: if you pretend that the risks can be brushed off as little more than the equivalent of an advertising jingle, you’re letting the AI labs off the hook.The AI industry isn’t just making the present worse; its recklessness may lead to the future of life on earth being catastrophically worse than the decidedly not-great future we can already anticipate.That seems to me like something that all progressives should care about.
Become a Sustaining Donor This September
Between the reemergence of right-wing red-baiting and Donald Trump’s use of the chilling National Security Presidential Memorandum–7 to criminalize left political organizing, we’re seeing McCarthyism reborn before our eyes.
This assault on democracy cannot be ignored or wished away.
Resisting it requires bravery in the face of threats and repression, a steadfast commitment to the truth, and the unwavering belief that we can—and will—overcome this darkness.
We must have an independent media that sets the record straight, unearths corruption and abuses of power, and advances a politics of justice and dignity for all if we’re to win this fight.
That’s why I’m writing to you today.This September, The Nation needs to add 100 monthly donors to sustain our progressive journalism.Your contribution makes the next investigative report, the next truth-speaking column, and the next pathbreaking essay all possible.
I hope you’ll support our work with a recurring donation today.If you donate $10 or more a month, we’ll send you a brand new “Hands Off the Free Press!” sticker to recognise your commitment to our cause.
Please, donate today.
Onward,
Katrina vanden Heuvel
Editor and Publisher, The Nation
More from The Nation
Yale’s President Has Become a Shameful Collaborator With Trump
Yale’s President Has Become a Shameful Collaborator With Trump
Maurie McInnis has the power to stand up to the administration.Instead, she’s caving.
The Fissures and Fractures Behind the Right’s Pink Facade
The Fissures and Fractures Behind the Right’s Pink Facade
After Roe was overturned, some young women are turning more hard-line on abortion—but others are fleeing conservatism altogether.
Notes From Evictionland
Notes From Evictionland
Eviction is practically baked into the way Britain manages its tenanted homes.Here’s how it played out—over and over again—for my parents.
How 9/11 Became a Joke
How 9/11 Became a Joke
The 25-year journey from national tragedy to national punchline.
We Can’t Let the Kaepernick Era Be Erased From History
We Can’t Let the Kaepernick Era Be Erased From History
Critics say the Black Lives Matter movement in sports went too far.
In reality, it didn’t go far enough, and the backlash to it has been brutal.
The Bund and the Radical History of Jewish Anti-Zionism
The Bund and the Radical History of Jewish Anti-Zionism
Molly Crabapple’s Here Where We Live Is Our Country revisits the past of the Jewish Labor Bund and considers what it might mean for our future..
