# @stem_antics on Instagram

- **Type:** Video
- **Original URL:** https://www.instagram.com/p/DbhjvU9gJ85
- **Gondola URL:** https://gondola.cc/posts/68532549-stem-antics-instagram
- **Thumbnail:** https://img.gondola.cc/tr:w-,h-,fo-auto/postThumbnails/192c5e1515.jpg
- **Posted:** 2026-08-02T04:30:20.000+00:00
- **Account Owner:** Stem Antics (@stem_antics) — https://gondola.cc/stem_antics

## Caption

Most robots learn to walk by guessing.

Standard reinforcement learning treats the physics engine as a sealed box. Push actions in, read a reward out, and estimate which way to nudge the policy by sampling thousands of noisy variations. It works. It is also brutally inefficient, often demanding hundreds of millions of simulated timesteps before a legged robot stops falling over.

A differentiable simulator removes the guessing. Instead of only computing the next state, it computes the derivative of that state with respect to every input that produced it. The simulator becomes a function you can backpropagate through, like any neural network layer.

What that unlocks:

- Analytic policy gradients rather than sampled estimates, cutting sample counts by one to three orders of magnitude on some benchmarks
- System identification by gradient descent, tuning friction, link masses and motor constants until simulated trajectories match real logged ones
- Design optimisation, where the geometry of the robot itself becomes a learnable parameter

The obstacle is contact. Rigid body collision is non-smooth. A foot either touches the ground or it does not, and the derivative at that boundary is undefined or infinite. Naive autodiff through a hard contact solver yields gradients that are technically correct and practically useless.

The workarounds are ingenious. Compliant contact models replace the discontinuity with a stiff spring. Randomised smoothing averages gradients over perturbed rollouts to recover a usable descent direction. Implicit differentiation extracts derivatives from the solution of the contact problem rather than from the solver iterations.

Long horizons bring a second problem. Chaotic dynamics amplify gradients exponentially, so most frameworks truncate the backward pass to short windows and hand the rest to a learned value function.

Guessing is expensive. Knowing which way to move is cheap.

#stemantics #robotics #machinelearning #simulation #engineering

## Stats

- **Views:** 8,367
- **Likes:** 216
- **Shares:** 0
- **Comments:** 2

## Tags

engineering, robotics, machinelearning, simulation, stemantics

---
Copyright (c) Gondola