What happens when war gets automated?
When we published our report on lethal autonomous weapons systems four years ago, our house view was that such systems were not just worth worrying about, but worth taking action on. We thought they were just over the horizon.
The horizon has moved closer. With the news, in April, of the establishment of the U.S. military’s new Autonomous Warfare Command, “slaughterbots” are no longer just a distant spectre but a permanent part of military conflict.
It’s been an eventful four years. In our report, we cited the Metaculus forecast of the chances that an autonomous assassination would occur by 2025. This forecast resolved “no” in January 2025. But it’s been a close one: slaughterbots are a regular feature of Ukraine’s battlefields, and there’s now early reports of a new type of Ukrainian drone that locks onto Russian soldiers’ faces and then fires.
Our report also cited Anthony Aguirre’s estimate that AI could drive “cost per fatality” in armed conflict below $400 — more morbidly cost-effective than even nuclear weapons. Ukraine’s drones are almost there: they can evidently kill a Russian soldier for around $1,000.
The incorporation of AI into so-called “kill chains” at a scale much more destructive than drone combat has dramatically accelerated the pace of warfare. In Gaza, an AI-enabled “mass assassination factory” enabled the Israeli military to identify and strike targets at a pace hundreds of times faster than ever before.
These weapons are also now, for the first time, a feature of political and public discourse. At the center of the Pentagon/Anthropic spat was Anthropic’s policy against allowing AI to kill without a human in the loop.
There have also been positive signs. President Biden and Chinese President Xi Jinping publicly resolved to keep humans in control of nuclear launch, a posture that made it into US law a few weeks afterward. Founders Pledge is proud to have funded the US/China AI dialogues that preceded these important steps via our Global Catastrophic Risks Fund.
There’s more to be done; our report also predicts a “race to the bottom” on safety and a trend toward catastrophically fast battlefield decisionmaking caused by a reliance on AI — a prediction that may already have already come true for the US military’s actions in Iran.
Perhaps the most concerning portent for the future is the idea that the low cost of automated death will not stay confined to the battlefield. As of this writing, Metaculus forecasts a 75% chance that more than 50 people will be killed by slaughterbots before 2035 — outside of a military conflict.
What can be done? Plenty, though none of it glamorous. When we fund Track II dialogues to bring Chinese and American representatives together, we are paying for coffee, crullers, and conference rooms. When we fund new mechanisms to control nuclear escalation, we are buying out teaching time from professors who study distant conflicts from within university walls. But it all matters. The Biden/Xi commitment on nuclear launch didn’t appear from nowhere: it was the product of years of patient, often boring work.
The surface area of conflict is expanding wider than the institutions that are meant to govern it. Autonomy is now a part not just of warfare but of military doctrine. Every incentive of the world’s existing institutions is to push further in this direction. It’s our responsibility to push against it.
