Late Sunday, 116 entrepreneurs, including Elon Musk, released a letter to the United Nations warning of the dangerous “Pandora’s Box” presented by weapons that make their own decisions about when to kill. Publications including The Guardian and The Washington Post ran headlines saying Musk and his cosigners had called for a “ban” on “killer robots.”
Those headlines were misleading. The letter doesn’t explicitly call for a ban, although one of the organizers has suggested it does. Rather, it offers technical advice to a UN committee on autonomous weapons formed in December. The group’s warning that autonomous machines “can be weapons of terror” makes sense. But trying to ban them outright is probably a waste of time.
That’s not because it’s impossible to ban weapons technologies. Some 192 nations have signed the Chemical Weapons Convention that bans chemical weapons, for example. An international agreement blocking use of laser weapons intended to cause permanent blindness is holding up nicely.
Weapons systems that make their own decisions are a very different, and much broader, category. The line between weapons controlled by humans and those that fire autonomously is blurry, and many nations—including the US—have begun the process of crossing it. Moreover, technologies such as robotic aircraft and ground vehicles have proved so useful that armed forces may find giving them more independence—including to kill—irresistible.
Related Stories
A recent report on artificial intelligence and war commissioned by the Office of the Director of National Intelligence concluded that the technology is set to massively magnify military power. Greg Allen, coauthor of the report and now an adjunct fellow at nonpartisan think tank the Center for New American Security, doesn’t expect the US and other countries to be able to stop themselves from building arsenals of weapons that can decide when to fire. “You are unlikely to achieve a full ban of autonomous weapons,” he says. “The temptation for using them is going to be very intense.”
The US Department of Defense does have a policy to keep a “human in the loop” when deploying lethal force. But it hasn’t suggested it would be open to international agreement banning autonomous weapons. The Pentagon did not immediately respond to a request for comment Monday. In 2015, the UK government responded to calls for a ban on autonomous weapons by saying there was no need for one, and that existing international law was sufficient.
You don’t have to look far to find weapons already making their own decisions to some degree. One is the AEGIS ship-based missile and aircraft-defense system used by the US Navy. It is capable of engaging approaching planes or missiles without human intervention, according to a CNAS report.
Other examples include a drone called the Harpy, developed in Israel, which patrols an area searching for radar signals. If it detects one, it automatically dive-bombs the signal’s source. Manufacturer Israeli Aerospace Industries markets the Harpy as a “‘Fire and Forget’ autonomous weapon.”
Musk signed an earlier letter in 2015 alongside thousands of AI experts in academia and industry that called for a ban on offensive use of autonomous weapons. Like Sunday’s letter, it was coordinated by the Future of Life Institute, an organization that ponders long-term effects of AI and other technologies, and to which Musk has gifted $10 million.
WIRED couldn’t reach the institute to ask why the new letter took a different tack. But Rebecca Crootof, a researcher at Yale Law School, says people concerned about autonomous weapons systems should consider more constructive alternatives to campaigning for a total ban.
“That time and energy would be much better spent developing regulations,” she says. International laws such as the Geneva Convention that restrict the activities of human soldiers could be adapted to govern what robot soldiers can do on the battlefield, for example. Other regulations short of a ban could try to clear up the murky question of who is held legally accountable when a piece of software makes a bad decision, for example by killing civilians.