A research paper on robotics authored by computer science researchers at Oregon State University was recently named one of the top four out of more than 2,000 accepted submissions at a prestigious conference.
Students Jonah Siekmann and Yesh Godse presented their research findings at the 2021 IEEE International Conference on Robotics and Automation. In their paper, “Sim-to-Real Learning of All Common Bipedal Gaits via Periodic Reward Composition,” they report on their work using simulations to teach two-legged robots how to run, skip, and hop.
The paper is co-authored with Alan Fern, professor and associate head of research in the School of Electrical Engineering and Computer Science, and Jonathan Hurst, professor of mechanical engineering and robotics.
Traditionally, researchers have tried to train bipedal robots to move by first creating a “reference trajectory,” which tells the robot at each moment where its joints and velocities should be. This approach, however, doesn’t work particularly well since it is difficult to figure out the reference trajectories, and it doesn’t take into account the uneven surfaces the robot needs to deal with.
Instead, the researchers’ new approach trains the robot in simulation, and rewards the robot when it is accomplishing the goal, and gives negative rewards when it is not.
“We use an approach that simply specifies constraints on the foot forces and velocities which allows us to specify the different types of gaits and smoothly move between them,” Fern said. “This worked much better than we ever expected.”
Siekmann, a master’s degree student in robotics who earned an honors bachelor’s degree in computer science from Oregon State in 2020, provided some additional insights.
“We were trying to train a neural network to learn various bipedal behaviors from scratch without any kind of motion capture or reference to what those behaviors looked like,” Siekmann explained. “To do this, we used deep reinforcement learning that allows a neural network to maximize a reward function.”
Added Godse, “It turned out that there was a simple mathematical framework for describing the full spectrum of all bipedal gaits and their corresponding reward/cost functions.”
Godse graduated in just three years with a bachelor’s degree in computer science from Oregon State in spring 2021 and began working on robotics research as a freshman.
Both Siekmann and Godse are now working as controls engineers at Agility Robotics, the company co-founded by Hurst that develops the robots used in Oregon State’s Dynamic Robotics Lab.