Leo and Phides – two planar biped robots built at the Delft University of Technology – are walking and running, respectively. Leo improved its walking gait through reinforcement learning, which shapes behavior by rewarding success and punishing failure. Phides, the running robot, has achieved an impressive flight phase (the period in a running stride in which both feet are off the ground).
Reinforcement learning (RL) normally favors computer simulation, not only because the hundreds (if not thousands) of trials can be run faster, but because real hardware would likely break down from the stress. Leo was built to test how RL would work in the real world. "Too little is yet known about the compatibility with real, actual hardware," explains PhD candidate Erik Schuitema. "It is a strength of TU Delft that it is able to create that very combination."
To prevent Leo from falling apart, Schuitema gave it a head start by providing it with a walking gait that it could learn from and improve upon. Leo's task is further simplified with the help of a stabilizing boom, which keeps its hip axis horizontal, but doesn't prevent the robot from falling forward or backward. Thanks to the boom, it needs just three servos per leg (rather than the usual six). All of its computing is handled by the robot's on-board computer, which has a 1.2 GHz VIA processor and 1 GB of RAM running Linux.
"If the hardware could be made more knock-proof, a robot like Leo could indeed learn to walk all by itself," Schuitema said. The process would likely take about five hours. "The work is still in its very early days. It could still be decades before these kinds of robots can get busy in practice," he added. The hope is that – some day – service robots would learn how to function in the real world on their own, without having to be programmed at all. Another approach is teaching by demonstration, which allows even non-programmers to train a robot to perform simple tasks.
Leo learns to walk
Researchers at the same lab have also developed a running biped robot called Phides, that can reach speeds of 2.2 mph (3.5 km/h) with a flight phase of 54 percent of each stride. This one didn't use reinforcement learning, but it too relies on a boom for horizontal stabilization, making it a two-dimensional robot similar to the University of Michigan's MABEL.
Due to size differences, it's difficult to compare it to MABEL: Phides stands a mere 31 inches (80 cm) tall and weighs 29 pounds (13.5 kg). "The key design feature of this robot is the actuation mechanism for the knees," the team writes in the YouTube video description. "Phides' knees have a series elastic actuator as well as a parallel spring ... constructed of fiberglass leaf springs from a crossbow" – which has got to be a world first!