How Robots Learn to Walk: Reinforcement Learning
The robot tries moves, gets a score for good results and learns by trial and error, mostly in simulation first.
The robot repeats this loop many times a second. Tap a step to pause on it.
In reinforcement learning a program tries actions and receives a reward when things go well, such as moving forward without falling. Over millions of tries it learns which actions give more reward.
Training in a simulator is fast and safe. The learned walking is then moved to the real robot, which is called sim-to-real.
A robot can discover a walking style that would be very hard to write by hand, and adapt to bumps and slopes.
Engineers must write every gait rule by hand and tune it again for each new robot or ground type.
Simulators are never perfect, so the real robot may behave differently, and a badly chosen reward can lead to odd cheating behaviour.
Why is training done in simulation first?
Go deeper in the Atlas
Control and Balance
- Feedback Control Explained
- Line Follower Robot
- Why You Don't Fall in a Moving Bus (And Robots Do)
- How Boston Dynamics Robots Keep Their Balance
- How Drones Stay Level in Strong Wind
- PID Controller in 60 Seconds
- ROS 2
- How Robots Learn to Walk
- A* Pathfinding
- State Machine
- Robot Latency
- Reward Hacking
- Sim to Real
- How Humanoid Robots Walk Without Falling (ZMP Explained)
- Gain and Damping
- How Self-Driving Cars Think
- Swarm Robotics
- Robot Dog (Quadruped) Explained
- How a Drone Hovers
- Behavior Tree