"Paradigm shift" in robotics: Alter3 robot combined with GPT-4: now it can take selfies and play an imaginary guitar and is programmed by voice instead of complex commands

A team of researchers from the University of Tokyo were able to build a link between the GPT-4 large language model and the Alter3 robot, allowing them to perform more human-like gestures and program the robot in natural language, abandoning traditional hardware-dependent control systems. The project authors called this progress a "paradigm shift" in robotics.

Alter3 is the latest version of the humanoid robot, which was first put into operation in 2016. It is capable of performing complex upper body movements, including detailed facial expressions, and has 43 axes that mimic the movement of the human musculoskeletal system. It rests on a base but cannot walk (although it can simulate walking). Using GPT-4, the researchers taught the robot to perform various simulations, such as taking a selfie, throwing a ball, eating popcorn, and playing an imaginary guitar. Previously, these activities required specific instructions to be coded for each action, but with GPT-4, robots can learn ordinary speech instructions.

The researchers state that "direct control is now possible by transferring the linguistic expressions of human actions to the robot body through software code. This allows for greater opportunities for human-robot collaboration. With this breakthrough in robotics, we can expect more intelligent, adaptive and personal robotic entities. The Alter3 robot has a built-in camera that allows it to "see" people and improve its behaviour by observing their reactions.

Through the use of language models in robotics research, scientists greatly simplify the process of coordinating the many joints of a robot by providing verbal instructions, describing movement options and obtaining the software code that starts the robot's engine. This eliminates the mundane task of programming each individual movement for 43 axes and allows researchers to reconfigure and refine the robot's movements more quickly, smoothly, and accurately over time.

The researchers also added a little humour to Alter3's activities. In one scenario, the robot demonstrates that it is eating a bag of popcorn, but then it turns out that the popcorn belongs to a person sitting next to it. The robot's facial expressions and hand movements convey surprise and shame.

Source: techxplore.com