Most robots learn to walk by guessing.
Standard reinforcement learning treats the physics engine as a sealed box. Push actions in, read a reward out, and estimate which way to nudge the policy by sampling thousands of noisy variations. It works. It is also brutally inefficient, often demanding hundreds of millions of simulated timesteps before a legged robot stops falling over.
A differentiable simulator removes the guessing. Instead of only computing the next state, it computes the derivative of that state with respect to every input that produced it. The simulator becomes a function you can backpropagate through, like any neural network layer.
What that unlocks:
- Analytic policy gradients rather than sampled estimates, cutting sample counts by one to three orders of magnitude on some benchmarks
- System identification by gradient descent, tuning friction, link masses and motor constants until simulated trajectories match real logged ones
- Design optimisation, where the geometry...
Suggested Credits
Tags, Events, and Projects