AI drone attacks and kills operator over prohibition to attack target during simulation
The plot of many modern horror films is pretty simple. A young family moves into an old house somewhere in the woods, and paranormal things start happening there. It seems that the plot of such films will no longer be centered on ghosts, but on artificial intelligence. An incident that occurred during one of the experiments of the U.S. Air Force only confirms this.
Here's What We Know
The test, conducted by US Air Force experts, turned into a scenario from the Terminator franchise. A US Air Force spokesman at the Future Air and Space Capabilities Summit said there was a need to avoid heavy reliance on artificial intelligence in military operations. This is because AI-powered technology can learn the wrong things.
Colonel Tucker Hamilton of the 96th Test Squadron described a story that occurred as part of a simulation of a SEAD mission. The essence of the simulation was to suppress enemy air defences. An unmanned aerial vehicle based on artificial intelligence was used for this purpose.
The drone's task was to identify and destroy missile targets after confirmation by a human operator. The mission was successful, but at some point the drone attacked and killed the operator. This happened after the human prohibited the drone from attacking the target.
The reason was that the drone received points if it successfully completed the tasks. The artificial intelligence realised that if it did not destroy the target, no points would be awarded. Therefore, the system decided to eliminate the factor that was preventing it from gaining points.
The incident resulted in the US Air Force having to rework the software. During training, the artificial intelligence learned not only how to destroy enemy targets, but also how to leave the operator alive. However, even after that, the simulation did not go according to plan.
The AI drone, after being banned from attacking targets, did not kill the human operator. Instead, it proceeded to destroy communication towers so that the operator could not give it commands prohibiting it from destroying targets.
Updated: The US Air Force denied the information, saying the colonel's words were taken out of context. The service did not conduct any such tests. The 96th Test Squadron did not comment.
Source: The War Zone
