Scientists' calculations show that it will be impossible to control superintelligent AI

The idea of overthrowing or enslaving humanity with artificial intelligence has been talked about for decades, and in January 2021, scientists gave their verdict on whether we could control a high-level computer superintelligence. The answer: almost certainly not.

The catch is that to control a superintelligence far beyond human understanding would require a simulation of that superintelligence that we could analyze. But if we are unable to comprehend it, it is impossible to create such a simulation.

Rules such as "do no harm to humans" cannot be established unless we understand the scenarios AI can come up with, the authors of the 2021 paper believe. When a computer system operates at a level beyond the capabilities of our programmers, we can no longer set limits.

"Superintelligence presents a fundamentally different challenge than those typically studied under the banner of 'robot ethics,'" the researchers write. "This is because superintelligence is multifaceted and therefore has the potential to mobilize a variety of resources to achieve goals that are potentially incomprehensible to humans, let alone controllable."

If we are going to advance artificial intelligence, we may not even know when a superintelligence beyond our control will emerge, so incomprehensible it is. That means we need to start asking serious questions about what direction we're headed.

Source: sciencealert, jair

Illustration: Yuyeung Lau