MechanismFrom first principles

Start with the smallest possible robot: a cart on a rail that has to stop on a mark. Everything true about the cart at one instant (its position, its velocity, the friction under its wheels, the wind pushing on it) is its state, $s_t$. The state is complete by definition: if you knew it exactly, and knew the physics, you could predict the next instant.

The robot never gets the state. It gets an observation, $o_t$: whatever its sensors report. A wheel encoder reports position, slightly noisy and a few milliseconds late. Nothing reports the friction or the wind. The observation is a partial, delayed and noisy view of the state.

Each tick the robot emits an action, $a_t$, here a motor command. A policy, $\pi$, is the rule that picks it. It can be a formula, a lookup table or, in everything that follows, a neural network. It can be deterministic, $a_t=\pi(o_t)$, or a probability distribution you sample from, $a_t\sim\pi(a\mid o_t)$. Because one observation rarely pins down the state, practical policies read a short history of observations, $\pi(a_t\mid o_{t-k},\dots,o_t)$.

Run a policy from a reset until something ends the attempt (a fall, a timeout, success) and you get a trajectory or episode, $(o_0,a_0,o_1,a_1,\dots)$. Producing one is called a rollout.

There are two ways to use this loop. Open loop means planning the whole action sequence in advance and playing it back blind. Closed loop means recomputing each action from the latest observation, so that errors are noticed and corrected.

The last toggle makes a point that recurs throughout the guide. Feedback is only as good as the observation it feeds on, and every observation is late and noisy. Most of the engineering in this field is about which parts of the loop are closed, at what rate, and with which sensors (A4, A5).

MechanismFrom first principles

A degree of freedom (DoF) is one independent way something can move. A door has one: the hinge angle. A rigid body floating in space has six: three for position and three for orientation. Each motorized joint adds one actuated DoF.

A fixed-base arm bolted to a table is easy to reason about. Its base never moves and every DoF is a joint with a motor, so commanding seven joint angles commands everything.

A humanoid is different. A Unitree G1 has 23 to 29 actuated body joints, and up to 43 with dexterous hands. Boston Dynamics’ electric research Atlas has about 50 DoF including its grippers; the production Atlas shown in January 2026 has 56. On top of the joints, the pelvis has six DoF in the world (x, y, z, roll, pitch, yaw) with no motor attached. This is the floating base, and the full configuration is “joints + 6”.

Nothing connects the pelvis to the world except contacts. To accelerate the body the robot must push on the environment, usually through its feet, and let the reaction forces move it. Newton’s law for the center of mass makes this concrete:

$$m\,\ddot p_{\text{com}} = \sum_i f_i + m\,g$$

The only terms the robot controls are the contact forces $f_i$, and the ground limits them. It can push but not pull, and it can push sideways only as far as friction allows. Each contact force must stay inside a friction cone:

$$f_z \ge 0, \qquad \sqrt{f_x^2+f_y^2} \le \mu\, f_z$$

Any force outside the cone is physically unavailable, however strong the motors are.

This is why legged control is harder than arm control. A fixed arm commands every DoF directly. A humanoid must choose where and how it touches the world so that the reaction forces do what it wants, and a bad step can cost it control of its six most important coordinates.

MechanismFrom first principles

Every motor ultimately produces torque. The simplest controller that makes a joint go somewhere acts like a virtual spring and damper:

$$\tau = K_p\,(q^{*}-q) + K_d\,(\dot q^{*}-\dot q)$$

Here $q$ is the measured joint angle, $q^{*}$ is the target, and $\dot q^{*}$ is usually zero. $K_p$ is a stiffness, torque per radian of error, like a spring anchored at the target. $K_d$ is a damping, torque per unit of velocity, like a shock absorber. This is PD control. The integral term of full PID control is usually left out for joints because it “winds up” when a limb is blocked by contact.

Seen this way, a policy that outputs $q^{*}$ isn’t commanding a position. It chooses where to anchor the spring, and the gains decide how hard the joint pulls. This is impedance control: controlling the relationship between motion and force rather than either one alone.

A worked example. Let $K_p = 40$ N·m/rad. The joint is at rest, 0.1 rad short of its target, so it produces 4 N·m. If the limb carries a load that needs 6 N·m to hold, the joint settles where $K_p(q^{*}-q)=6$, which is 0.15 rad below the target. A policy trained with this controller learns to aim above where it wants the joint to end up.

Four reasons nearly every learned locomotion and whole-body policy outputs position targets:

1. Disturbance rejection between ticks. The PD loop runs at about 1 kHz on the motor driver; the policy runs every 20 ms. A kick at millisecond 3 is resisted immediately, not 17 ms later.
2. Latency tolerance. If the next target arrives late, the joint keeps servoing to the last one instead of going limp.
3. Safe exploration. RL explores by adding noise to actions. Noise on targets makes a joint jiggle around a pose; noise on torques integrates into runaway motion.
4. Hiding actuator ugliness. Friction, backlash and current limits sit inside a feedback loop, which shrinks the gap between simulation and reality (B6).

The gains are part of the action space. The same target produces different motion under different gains, so changing $K_p$ and $K_d$ after training changes the policy’s behavior.

Torque control, where the policy outputs $\tau$ directly, is more expressive. It can be exactly compliant or apply a precise force, and it avoids the artificial stiffness a PD loop imposes. But it needs a policy running at a very high rate, its exploration is dangerous, and the simulator must model torque tracking accurately. Figure’s S0 whole-body controller emits joint-level actuator commands at 1 kHz, which is possible only because the network is tiny, about 10 million parameters (C6).

MechanismFrom first principles

Physical events happen on very different timescales. A foot strikes the ground in a few milliseconds. A step or a reach takes a few hundred. Wiping a counter takes seconds, and loading a dishwasher takes minutes.

A control loop has to run several times faster than the dynamics it shapes; a common rule of thumb is five to ten times. At the same time, computation per tick grows with model size. A forward pass through a 3-billion-parameter model on an onboard GPU takes tens to hundreds of milliseconds, while a 10-million-parameter network takes well under one. So nothing can be both big and fast, and stacks are layered by rate:

The pattern is consistent. The faster the loop, the smaller the model and the more local its job; the slower the loop, the bigger the model and the more it knows.

MechanismFrom first principles

Proprioception is internal sensing. Joint encoders report each joint’s angle, and velocity by differentiation, usually at 1 kHz or faster. An inertial measurement unit (IMU) in the pelvis or torso reports angular velocity and linear acceleration, from which orientation relative to gravity is estimated. Motor current gives an estimate of joint torque, good on low-gear-ratio actuators and poor on high-ratio ones (I1). Foot force sensors, where present, report contact.

Exteroception is sensing the outside world: RGB cameras, depth cameras, lidar. Touch sits between the two. Fingertip tactile sensors measure contact the way cameras can’t: Figure 03’s fingertips register forces as small as 3 grams (G8).

The two families differ in more than subject. Proprioception is low-dimensional (a few dozen numbers), fast, and nearly noise-free. Cameras produce hundreds of thousands of numbers per frame at 30–60 Hz, and those numbers are hard to interpret.

“Blind” locomotion uses proprioception alone, and it works surprisingly well. A short history of joint states and IMU readings implicitly reveals the terrain and the robot’s own dynamics. If a foot touches down earlier than commanded, the ground is higher than expected; if a joint lags its target, the load is heavier. A policy with memory can infer these hidden quantities without seeing anything (B6).

MechanismFrom first principles

A learned policy is fitted to a training distribution: the range of scenes, objects, instructions and physical conditions in its data. On situations that resemble that data (in-distribution) it interpolates between things it has seen. On situations outside it (out-of-distribution) it has to extrapolate, and learned models extrapolate badly.

Papers describe test conditions as seen (in the training data) or unseen (absent from it), usually along a named axis:

• Visual: new lighting, backgrounds, camera positions.
• Object: new instances or categories of objects.
• Scene: new rooms, homes or factories.
• Semantic: new phrasings or instructions (“the extinct animal”).
• Task: new combinations of known skills, or new skills.
• Embodiment: a robot body the model never trained on (D9).

Zero-shot means succeeding with no task-specific training at all. Few-shot means adapting from a handful of examples or a short fine-tune (D11).

The figure shows the core difficulty. A model can have near-zero error on everything like its training data and be arbitrarily wrong a small step beyond it. Neural networks behave better than high-degree polynomials, but the principle holds: their guarantees stop at the edge of the data.

Defining “unseen” is getting harder as datasets grow. When a model has been pretrained on the internet and hundreds of thousands of hours of robot data, a “novel” kitchen may still resemble something in training. The π0.7 authors say as much. Careful papers therefore report which axes changed, and hold out whole environments rather than individual episodes (H1).

MechanismFrom first principles

Parameters are a network’s learned numbers (weights and biases). “A 3B model” has three billion of them.

An embedding or latent is a vector of numbers that represents something, such as an image patch, a word or a goal, in a space where similar things end up near each other. Networks do their work on embeddings.

A token is one item in a sequence that a transformer processes: a word piece, an image patch, a discretized action (D8). Everything the model reads or writes is converted to tokens and embedded.

Attention lets each token build its representation by mixing information from other tokens. Each token computes a query $q$, a key $k$ and a value $v$ (three linear projections of its embedding). Token $i$ weights token $j$ by how well its query matches $j$’s key:

$$\text{Attention}(Q,K,V) = \operatorname{softmax}\!\Big(\frac{QK^{\top}}{\sqrt d}\Big)\,V$$

A transformer stacks layers of attention and small per-token MLPs. Because every token can attend to every other, cost grows with the square of sequence length, which is why token counts matter for robot latency (D6).

A ViT (vision transformer) chops an image into patches, typically 14 or 16 pixels square, embeds each patch as a token, and runs a transformer over them. A 224×224 image in 14-pixel patches becomes 16 × 16 = 256 tokens. A DiT (diffusion transformer) is a transformer used as the denoiser inside a diffusion or flow model (D4).

The loss is the number training minimizes. Mean squared error (MSE) is used for regression, predicting continuous values. Cross-entropy is used for classification and next-token prediction. Validation loss is the loss on held-out data, used to measure generalization without running the robot; it is only loosely coupled to real task success (G6).

Fine-tuning continues training a pretrained model on new data. Freezing locks some weights so they don’t change. LoRA (low-rank adaptation) freezes a weight matrix $W$ and learns a small correction $W + BA$, where $B$ and $A$ are thin matrices of rank $r$; for a $d\times d$ matrix it trains $2dr$ numbers instead of $d^2$ (D11). Distillation trains a student model to reproduce a teacher’s outputs (B7, E6).

MechanismFrom first principles

RL formalizes control as a Markov decision process (MDP): states $s$, actions $a$, transition dynamics $p(s'\mid s,a)$, and a scalar reward $r_t$ received each step. The policy’s goal is to maximize the expected return, the discounted sum of future rewards:

$$G_t = \sum_{k=0}^{\infty} \gamma^{k}\, r_{t+k}$$

The discount $\gamma$, typically 0.99, makes the effective planning horizon about $1/(1-\gamma)$ steps. At $\gamma = 0.99$ that is 100 steps, which is 2 seconds for a policy running at 50 Hz. Rewards further away still count, but less.

Three quantities organize everything else:

• The value function $V(s)$: the expected return from state $s$ if you follow the policy.
• The action-value $Q(s,a)$: the expected return from taking action $a$ in $s$, then following the policy.
• The advantage $A(s,a) = Q(s,a) - V(s)$: how much better this action was than the policy’s average from here.

Values obey a recursive consistency condition, the Bellman equation: $V(s) = \mathbb{E}\,[\,r + \gamma V(s')\,]$. The value of a state is the reward you get now plus the discounted value of where you land. Most RL algorithms either solve this equation approximately, or use it to score actions.

The policy gradient turns this into learning. Increase the log-probability of actions with positive advantage and decrease it for actions with negative advantage:

$$\nabla_\theta J = \mathbb{E}\big[\,\nabla_\theta \log \pi_\theta(a\mid s)\; A(s,a)\,\big]$$

Actor-critic methods train two networks: the actor is the policy, and the critic estimates $V$ so advantages can be computed. On-policy algorithms (PPO, TRPO) learn only from data collected by the current policy and throw it away after each update. Off-policy algorithms (SAC, TD3, DQN) keep a replay buffer and reuse old data. For continuous actions, exploration usually means adding Gaussian noise: the policy outputs a mean and a standard deviation, and actions are sampled.

MechanismFrom first principles

Vanilla policy gradients have a sample problem. A batch of rollouts is expensive, so you want to take several gradient steps on it. But after the first step the policy has changed, and the batch no longer reflects what the new policy would do. The fix is an importance ratio, which reweights each action by how much more or less likely the new policy makes it:

$$r_t(\theta) = \frac{\pi_\theta(a_t\mid s_t)}{\pi_{\theta_{\text{old}}}(a_t\mid s_t)}$$

Maximizing $r_t \hat A_t$ improves the policy, but nothing stops it from moving too far. The ratio becomes unreliable, updates overshoot, and performance collapses. TRPO (2015) solved this with a trust region, a hard limit on how far the policy distribution may move, enforced with second-order machinery. PPO (2017) gets most of the benefit with a clip:

$$L^{\text{CLIP}}(\theta) = \mathbb{E}_t\Big[\min\big(r_t(\theta)\,\hat A_t,\ \operatorname{clip}(r_t(\theta),\,1-\epsilon,\,1+\epsilon)\,\hat A_t\big)\Big]$$

With $\epsilon \approx 0.2$, the objective stops rewarding changes once an action’s probability has moved more than about 20% in the helpful direction. The gradient there becomes zero, so the update can’t profit from going further. The min makes the bound pessimistic: moves in the harmful direction are never clipped away.

The advantages come from generalized advantage estimation (GAE). Start from the one-step temporal-difference error, which says how much better a step went than the critic expected:

$$\delta_t = r_t + \gamma V(s_{t+1}) - V(s_t), \qquad \hat A_t = \sum_{l\ge 0} (\gamma\lambda)^{l}\, \delta_{t+l}$$

The parameter $\lambda$ trades bias for variance. At $\lambda = 0$ you trust the critic completely (low variance, biased if the critic is wrong). At $\lambda = 1$ you use the actual returns (unbiased, noisy). Locomotion typically uses $\lambda \approx 0.95$.

The training loop:

1. Run thousands of simulated robots for a few dozen steps each; Rudin et al. used 4,096 robots × 24 steps, a batch of 98,304 transitions.
2. Compute advantages with GAE from the critic’s values.
3. Run a few epochs of minibatch gradient ascent on $L^{\text{CLIP}}$, plus a critic regression loss and a small entropy bonus.
4. Throw the batch away and repeat.

MechanismFrom first principles

A locomotion reward is a weighted sum of terms, computed every step:

• Task terms reward what you want, usually as a smooth kernel of an error: velocity tracking as $\exp(-\lVert v^{*}-v\rVert^2/\sigma)$, or pose tracking (C3).
• Regularizers penalize what you don’t want: action rate (jerkiness), torque, joint acceleration, joint-limit violations, foot slip. Some reward good habits, such as feet spending time in the air, which encourages real steps rather than shuffling.
• A termination penalty for falling, often combined with ending the episode.

The weights span orders of magnitude. ASAP’s motion-tracking policy, for example, uses −10 for joint-limit violations, −5 for torque-limit violations, −0.5 for action rate and −200 for termination, alongside positive tracking terms.

Why so many terms? Because RL finds whatever the simulator rewards. Without regularizers it discovers exploits: vibrating joints at the control frequency, jamming feet into contact-solver artifacts, or gaits that rely on unrealistically perfect friction. Every regularizer is a patch that makes the policy transferable and mechanically survivable.

The exponential kernel matters too. Its width $\sigma$ sets how forgiving the reward is. A narrow kernel rewards only near-perfect tracking, which gives almost no gradient when the policy is far off; a wide one gives signal everywhere but little incentive for precision.

Curricula keep the problem inside a learnable band. If a task is impossible at first, every episode ends in failure and there is nothing to learn from; if it stays easy, the policy never learns the hard cases. Rudin et al.’s terrain curriculum puts each robot on a terrain level and promotes it when it walks far enough, demoting it when it fails. Command curricula widen the range of commanded velocities as the policy improves.

MechanismFrom first principles

A classic physics engine steps one scene on a CPU. A GPU-parallel simulator steps thousands of independent copies of a scene as one batched computation, and computes observations, rewards and resets as vectorized tensor operations. Crucially, it also runs the policy network on the same GPU, so no data crosses the slow CPU–GPU boundary during training.

Isaac Gym and Isaac Lab (built on PhysX), MuJoCo MJX (MuJoCo rewritten in JAX so it can be vectorized over environments), Brax, Genesis, Warp and Newton all work this way. Throughput is on the order of $10^5$–$10^6$ environment steps per second on one GPU. Rudin et al. trained ANYmal with 4,096 environments: flat-ground walking in under 4 minutes and rough terrain in about 20 on a single workstation GPU. Figure’s S0 trained in more than 200,000 parallel environments.

The arithmetic is simple. PPO for a whole-body skill may need $10^9$–$10^{10}$ samples. At $10^4$ steps per second, typical of CPU simulation, $10^{10}$ samples take about 12 days. At $10^6$ per second they take under 3 hours. Parallelism also makes domain randomization free: each of the 200,000 robots can have its own mass, friction and motor strength (B5).

MechanismFrom first principles

During training, each environment samples its physical parameters from a distribution: link masses and centers of mass, friction coefficients, motor strength and PD-gain scaling (A3), joint damping, action latency (commands delayed by a random number of steps), sensor noise, random external pushes, terrain shapes, and, for vision, textures, lighting and camera pose.

It works through two mechanisms.

Invariance. The policy is optimized for expected return across the whole distribution, so it learns behaviors that work for all of its members. This is conservative but robust.

Implicit system identification. If the policy sees a history of observations (B6), it can infer which parameters it is currently facing from how the robot responds to its own commands. A sluggish joint means a weak motor or a heavy load, and the policy adapts. Under this view the real robot is one more sample from a distribution the policy has already covered.

The toy captures the central trade-off. Too wide a range and the policy can’t learn, or becomes over-conservative; ASAP’s authors note that randomization can “sacrifice agility”. Too narrow, and reality falls outside it.

Randomization also has a hard limit. It can’t fix structural mismatch: phenomena the simulator has no knob for, such as gearbox backlash, cable drag, thermal derating, or contact behavior the solver can’t produce. Those need actuator models (B6) or learned corrections (B8).

MechanismFrom first principles

System identification comes first because it is cheap. Weigh the links, take masses and inertias from CAD, fit motor step responses, and estimate friction from simple tests, then set the simulator to match.

Actuator networks. ANYmal’s series-elastic actuators had torque responses no analytic model captured: friction, backlash, controller delay, current limits. Hwangbo et al. (2019) logged real actuator behavior and trained a small MLP (three hidden layers of 32 units) that maps recent position errors and velocities, at $t$, $t-0.01$ s and $t-0.02$ s, to the torque actually produced. They put that network into the simulator in place of an ideal actuator, and policies transferred almost perfectly. The actuator is the least well-modeled component and the one the policy touches most directly.

Privileged information and the asymmetric actor-critic. During training the critic may see the true state: friction, terrain heights, external forces, exact base velocity. The actor sees only what the real robot will have. This is legitimate because the critic only computes advantages during training and is discarded at deployment. A critic with full information estimates values with far less variance, so learning is faster and more stable.

History encoders. Feed the last N observations (for example 50 steps, one second) into the policy, by stacking them or through a GRU, a temporal convolution or a transformer. This supplies the raw material for the implicit system identification of B5, and for estimating quantities that are never measured directly, such as base velocity and foot contact. Radosavovic et al. deployed a causal transformer over observation history on the Digit humanoid, and it walked outdoors zero-shot. A follow-up treated the robot’s whole sensorimotor stream as a sequence to be predicted token by token, which lets one model train on heterogeneous data including motion capture and video.

Rapid Motor Adaptation (RMA) makes the adaptation explicit, in two phases. In phase 1, a policy is trained by RL with an extra input: a compact “extrinsics” vector $z$ produced by an encoder from the privileged parameters. In phase 2, an adaptation module is trained by supervised regression to predict $z$ from a history of observations and actions. At deployment the adaptation module replaces the encoder and re-estimates $z$ every step.

RMA’s two phases illustrate a pattern that recurs across this guide. RL with privileged input is easy; regressing a latent from history is easy; RL under partial observability directly is hard. So split the hard problem into two easy ones (B7).

MechanismFrom first principles

The paradigm has two stages.

1. Train a teacher by RL with privileged observations: full state, future reference frames, a terrain map. With everything visible the problem is nearly fully observed, so RL is tractable.
2. Train a student that sees only deployable inputs (proprioception, sparse VR trackers, a camera) to imitate the teacher’s actions by supervised learning.

Why not train the student with RL directly? Partial observability makes exploration and credit assignment brutally harder. The student would have to discover balance and infer hidden state at the same time. Distillation splits this into “solve a fully observed problem with RL” and then “solve a regression problem”.

The regression has to be done carefully. If the student only regresses on the teacher’s own trajectories, it drifts into states the teacher never visited and has no idea how to recover. This is the compounding-error problem of behavior cloning (D1). DAgger (dataset aggregation) fixes it:

1. Roll out the student.
2. At every state the student visited, ask the teacher what it would have done.
3. Add those (state, teacher action) pairs to the dataset.
4. Retrain the student on everything collected so far, and repeat.

Because DAgger labels the student’s own mistakes, the student learns recoveries. Distillation in sim is ideal for this: the teacher can be queried at any state, as often as needed, for free.

MechanismFrom first principles

Randomization handles parameter mismatch, but agile motion exposes structural mismatch the simulator has no knob for (B5). ASAP (He et al., 2025) attacks that directly, in four stages:

1. Pretrain motion-tracking policies in simulation on retargeted human motion (C3).
2. Deploy on the real Unitree G1 and record the real state trajectories and the actions sent.
3. Replay those same actions in simulation. The simulated trajectory diverges from the real one. Train a delta-action policy $\Delta a_t = \pi^{\Delta}(s_t, a_t)$, by RL, rewarded for making the simulated trajectory match the real one when the action is replayed as $a_t + \Delta a_t$.
4. Freeze $\pi^{\Delta}$ inside the simulator, so that $s_{t+1} = f_{\text{sim}}(s_t,\, a_t + \Delta a_t)$, and fine-tune the motion policy in this corrected simulator.

Why correct the action rather than the dynamics? Learning a full dynamics correction in state space is high-dimensional and can produce physically inconsistent states. A small residual in action space keeps the simulator’s physics intact, since every corrected rollout still obeys rigid-body mechanics, and is easy to learn.

ASAP reported tracking-error reductions of up to 52.7% in its sim-to-real experiments. One independent review has questioned how that figure was computed, so treat the magnitude with care and the mechanism as sound.

MechanismFrom first principles

Model predictive control (MPC) solves, at every control tick, an optimization over a short future horizon. It uses a simplified dynamics model, often a single rigid body or centroidal dynamics, which treats the robot as one mass with angular momentum. It chooses contact forces that keep balance and follow a plan subject to the friction cones of A2, executes the first step, and re-solves at the next tick. This is receding-horizon control.

A whole-body QP (quadratic program) then allocates joint torques at each instant to satisfy prioritized tasks, such as keeping the center of mass here and putting the foot there, subject to contact, friction-cone and torque-limit constraints:

$$\min_{\ddot q,\,\tau,\,f}\ \sum_i w_i \lVert J_i \ddot q + \dot J_i \dot q - \ddot x_i^{*}\rVert^2 \quad \text{s.t.}\quad M\ddot q + h = S^{\top}\tau + J_c^{\top} f,\ \ f \in \text{cone}$$

The equality constraint is the robot’s equations of motion, the Newton–Euler law of A2 extended to every joint. It is a convex problem, solvable in well under a millisecond.

The strengths are real: interpretable behavior, tunable weights, formal constraints and no training data. This is what powered Atlas parkour. The weaknesses are just as real. MPC needs an accurate model and hand-designed contact schedules that say when each foot touches down. Every new behavior is an engineering project, the computation per tick is heavy, and anything the model leaves out makes it brittle.

MechanismFrom first principles

Motion capture (mocap) turns human movement into numbers: the position and orientation of the body’s root (the pelvis) and the rotation of every joint, sampled 30–120 times a second. Three families of devices do it, trading accuracy against cost:

• Optical systems (Vicon and similar) put retroreflective markers on the body and triangulate them with a ring of infrared cameras. Millimeter-accurate, but confined to an instrumented studio.
• Inertial suits (Xsens and similar) put an IMU on each body segment and integrate orientations. Portable and camera-free, but position drifts over time.
• Video pose estimation (OpenPose for 2D keypoints, HMR and 4DHumans for full 3D bodies) infers the skeleton from ordinary video. Nearly free and available at internet scale, but noisy and ambiguous in depth.

Every lab uses different marker sets and skeletons, so pooling data requires a common body. SMPL (2015) provides one: a triangle mesh of 6,890 vertices controlled by two small sets of numbers. Pose parameters $\theta$ give the rotation of 24 joints (72 numbers as axis-angle). Shape parameters $\beta$, about 10 principal components, capture body proportions. The same $\theta$ on a different $\beta$ is the same movement performed by a different body.

AMASS (2019) fitted SMPL to 15 optical mocap datasets with a method called MoSh++, unifying them into one archive of about 40 hours, 344 subjects and 11,265 motions. LAFAN1 (2020, Ubisoft) added about 4.6 hours of production-quality animation data; the older CMU MoCap database is a longtime staple. Industry now captures far more: NVIDIA’s SONIC trained on more than 100 million frames (about 700 hours), and Figure’s S0 on over 1,000 hours of retargeted human motion.

MechanismFrom first principles

Kinematic retargeting usually proceeds in five steps:

1. Map corresponding joints between the human skeleton and the robot’s (human hip to robot hip, and so on).
2. Scale positions by limb-length ratios, so that a stride fits the robot’s legs.
3. Solve inverse kinematics (IK) frame by frame: find robot joint angles that put key points (feet, hands, head) where the scaled human’s are relative to the pelvis, while matching joint orientations where possible.
4. Respect limits: clamp to joint ranges and avoid self-collision.
5. Fix contacts: detect when feet are planted and pin them, removing foot skating, the sliding that appears when proportions differ.

It is harder than it sounds, for three reasons. Different proportions mean you can’t match both positions and joint angles at once. Different degrees of freedom mean some motions can’t be expressed: a human spine bends in many places while a robot torso may have one waist joint, and wrist and ankle ranges differ. And different mass distribution means many human motions demand torques or speeds the robot lacks, or center-of-mass paths its feet can’t support (A2).

The output of pure kinematic retargeting is a trajectory that looks right but is physically infeasible in places. Three remedies are in use:

• Physics-based filtering. H2O’s “sim-to-data” step trains a privileged tracker in simulation and discards clips it can’t track, leaving a feasible dataset.
• Optimization with physical constraints. Retargeting that enforces contact and dynamics directly. OmniRetarget (2025) preserves contacts with objects and terrain via an interaction mesh; GMR (2025) is a fast general retargeter and showed that retargeting artifacts measurably hurt the tracking policies trained on them.
• Learning the mapping. SONIC’s authors describe a universal token space that maps human motion from several sources into robot control commands without a separate per-clip retargeting stage.

MechanismFrom first principles

The policy observes proprioception plus a window of the reference motion: the current and next few hundred milliseconds of target joint angles, root pose and keypoint positions. It is rewarded for matching them:

$$r_t = w_p\, e^{-\alpha_p\lVert q^{\text{ref}}_t - q_t\rVert^2} + w_v\, e^{-\alpha_v\lVert \dot q^{\text{ref}}_t - \dot q_t\rVert^2} + w_e\, e^{-\alpha_e\lVert p^{\text{ref}}_t - p_t\rVert^2} + w_r\, e^{-\alpha_r\lVert x^{\text{ref}}_t - x_t\rVert^2}$$

The four terms reward joint pose, joint velocity, end-effector and keypoint positions, and root position and orientation, each through the exponential kernel of B3. The usual regularizers are added on top.

Two tricks from DeepMimic (2018) are essential:

• Early termination. End the episode as soon as the robot falls or deviates beyond a threshold. The policy never collects reward while lying on the floor, and simulation time isn’t wasted there.
• Reference state initialization (RSI). Start each episode at a random frame of the clip, in the reference pose. Otherwise the policy must master the beginning before it ever experiences the end, and long clips never get learned.

What does RL add beyond playing the retargeted motion back? The reference says where the body should be, not how to stay upright while getting there. The learned tracker discovers the balance corrections, weight shifts and foot-force adjustments that make a physically imperfect reference executable, and, trained with randomization (B5), executable on hardware.

MechanismFrom first principles

Unconstrained exploration is hopeless for a humanoid: random joint targets produce contorted poses and falls, so almost no episode yields useful reward. A prior restricts or biases exploration toward human-like, balanced motion. Priors come in four forms.

Explicit tracking. The prior is a specific clip, as in C3. It is precise, but the policy has no freedom to choose a different motion that would serve a task better.

Adversarial motion priors (AMP, 2021). Train a discriminator $D$ to tell mocap state transitions $(s, s')$ from the policy’s own transitions. The policy receives a style reward for fooling it. AMP’s version uses a least-squares discriminator and the reward

$$r^{\text{style}} = \max\big[0,\ 1 - 0.25\,(D(s,s') - 1)^2\big]$$

added to a task reward. The policy may now use any motion in the dataset that helps the task, with no clip alignment required. The costs are adversarial training’s instability and mode collapse, where the policy reuses a few favorite motions, and less control over which motion appears.

Latent skill spaces (ASE, PULSE). First train a universal tracker. PHC (2023) tracks 98.9% of AMASS by adding motor primitives one after another for progressively harder motions and composing them multiplicatively. Then distill it into an encoder-decoder, where a compact latent $z$ decodes to actions and a learned prior describes plausible $z$. A new task is then learned by RL over $z$ instead of raw joint targets. Every point in that space is a human-like, balanced motion, so exploration becomes dramatically easier and safer. The robot can do only what the prior covers.

Masked, partial conditioning (MaskedMimic, HOVER). Train one controller on randomly masked subsets of the full-body target: sometimes only a hand target, sometimes only root velocity, sometimes joint angles. The result accepts whatever command a higher layer can supply. The costs are distillation effort and slightly lower quality per mode than a specialist.

MechanismFrom first principles

ExBody (UCSD, 2024). The upper body tracks human motion; the legs track only a root-movement command (velocity, orientation and height), because human-like leg tracking transferred poorly to the Unitree H1. It showed expressive motion on hardware by not asking the legs to imitate.

H2O and OmniH2O (CMU, 2024). Real-time whole-body teleoperation: a person’s motion, from an RGB camera or VR, drives the humanoid through a tracking policy. H2O introduced the sim-to-data filter for infeasible motions (C2). OmniH2O distilled a teacher that saw the full reference into a student that sees only the head and hand poses a VR headset provides (B7).

HumanPlus (Stanford, 2024). A shadowing transformer tracks a human seen by one camera; humans then “shadow” to collect demonstrations, and an imitation transformer learns autonomous skills from them. Teleoperation and data collection become the same activity.

HOVER (NVIDIA, 2024). One policy accepts any combination of command modes (joint angles, keypoints, root velocity), distilled from an oracle with masked commands (C4).

BeyondMimic (Berkeley and Stanford, 2025). A unified tracking recipe produced sprints, spin-kicks and cartwheels on hardware; many trackers were then distilled into one guided diffusion policy that can be steered at inference by simple cost functions (joystick, waypoints, obstacle avoidance) to compose new tasks zero-shot.

SONIC (NVIDIA, 2025). Scale everything: models from 1.2M to 42M parameters, more than 100 million frames (about 700 hours) of motion, and tens of thousands of GPU-hours (the paper’s versions report between 9,000 and 32,000). The result is a motion-tracking foundation model with a universal token interface for teleoperation and VLAs.

Figure S0 (2026). A 10-million-parameter controller that maps full-body joint state and base motion to joint-level actuator commands at 1 kHz, trained on over 1,000 hours of retargeted human motion in more than 200,000 parallel environments. It replaced 109,504 lines of hand-written C++.

Two threads run through the list. One is the interface: what a higher layer, or a human, uses to command the body. It narrowed from full-body references to head-and-hands to learned tokens. The other is scale: from a handful of clips to a thousand hours, and from research code to a controller that ships.

MechanismFrom first principles

Definition. A learned whole-body controller is a policy that commands all of the robot’s joints (legs, torso, arms, sometimes hands) to achieve a commanded objective while maintaining balance, learned rather than derived from equations. “Whole-body” contrasts with separate leg and arm controllers, and with the optimization-based whole-body control of B9; “learned” is the other half of that contrast.

Inputs are proprioception (joint positions and velocities, IMU), a short history, and a command: a root velocity, arm joint targets, keypoint targets for hands and feet, a full reference pose, or a latent token. Outputs are joint position targets for the PD loops (A3) or, for S0, direct actuator commands at 1 kHz.

Training assembles most of this guide. Retargeted human motion (C2) supplies references. PPO (B2) in GPU simulation (B4) with randomization (B5) and tracking rewards (C3) produces a privileged teacher. Teacher-student distillation (B7) produces a deployable student, and delta-action alignment (B8) optionally closes the remaining gap.

What it buys is a clean action space for everything above it. A manipulation policy can say “right hand there, walk at 0.5 m/s” and never think about balance. It also defines the teleoperation interface (G2), and therefore the data distribution the upper layers learn from.

Decoupled or unified. Early systems split the body. ExBody, the decoupled controller shipped with NVIDIA’s GR00T N1.5 and N1.6 (RL for the legs, IK for the arms), and Helix’s first version (upper body only) all did, because legs needed robust RL while arms needed dexterous imitation. The cost is real, as Figure put it: “Lift something and your balance changes; step forward and your reach changes.” Helix 02, Boston Dynamics’ Atlas large behavior model (which commands hand and foot poses), 1X’s Redwood, SONIC-backed VLAs and Gemini Robotics 2 now emit whole-body commands and let a learned controller stabilize underneath.

The interface matters. Joint targets are maximally expressive but robot-specific, so only robot data can train the layer above. End-effector or keypoint poses are nearly embodiment-agnostic and can be computed from human video (G3). Latents (PULSE embeddings, SONIC tokens, Helix’s S2-to-S1 vector) are compact and enforce the prior, but are opaque. The interface decides which data can train the upper layers and how much they must know about balance.

MechanismFrom first principles

Behavior cloning (BC) fits a policy to recorded (observation, action) pairs by maximizing the likelihood of the expert’s actions, $\max_\theta \sum \log \pi_\theta(a\mid o)$, or, for continuous actions, minimizing squared error. It needs no reward and no simulator: only demonstrations.

Its weakness is covariate shift. Training data comes from states the expert visited. At test time the policy visits states produced by its own actions. Any error, even a small one, moves it slightly off the expert’s distribution, where its predictions are worse, which moves it further off. Ross and Bagnell (2010) made this precise. If the policy errs with probability $\epsilon$ per step in expert states, its expected cost over a horizon $T$ can grow as

$$J(\pi) \le J(\pi^{*}) + O(\epsilon\, T^{2})$$

compared with $O(\epsilon T)$ for a learner trained on its own state distribution, as DAgger does (B7). Doubling the task length can quadruple the damage.

Everything from action chunking (D3) to DAgger (B7) to RL post-training (Part E) is a response to this quadratic term.

MechanismFrom first principles

Demonstrations are multimodal. For the same observation, different demonstrators, or the same one on different days, go left or right around an obstacle, grasp a mug by the handle or the rim, fold the left sleeve first or the right. The conditional distribution of actions given the observation has several peaks.

A policy trained with mean squared error learns the conditional mean of that distribution, because the mean is what minimizes squared error. For a left/right split, the mean is “straight ahead”, which is straight into the obstacle. This is mode averaging, and it is why plain regression fails on real human demonstrations.

The fixes, in rough historical order, all model the full distribution and sample one coherent mode from it:

• Mixture density networks predict a Gaussian mixture. They work, but are awkward in high dimensions.
• Discretized actions turn regression into classification over bins, which can represent several peaks but is coarse (D8).
• Conditional VAEs add a latent “style” variable that selects the mode (ACT’s approach, D3).
• Energy-based models (implicit behavioral cloning) score (observation, action) pairs and search for low energy. Powerful, but hard to train.
• Diffusion and flow models, the current default (D4, D5).

MechanismFrom first principles

ACT (Action Chunking with Transformers, Zhao et al., 2023) predicts a chunk of $k$ future actions from the current camera images and joint state. The default is $k = 100$ at 50 Hz, two seconds of motion. A transformer encoder-decoder runs over ResNet image features.

To handle multimodal demonstrations (D2) it is a conditional VAE. During training an encoder maps the demonstrated action chunk to a latent “style” $z$, and the decoder predicts the chunk from the observations and $z$. At test time $z$ is set to the prior mean, which selects a single consistent style. On the low-cost ALOHA bimanual rig, ACT reached 80–90% success on fine tasks, such as opening a translucent condiment cup, with “only 10 minutes worth of demonstrations”.

Why chunking works:

1. Fewer decisions. A horizon of $T$ steps becomes $T/k$ decisions, directly attacking the $\epsilon T^2$ bound of D1.
2. Non-Markovian behavior. Humans pause, hesitate and correct. The next action is often unpredictable from the current frame, yet predictable as part of a sequence.
3. Amortized compute. One inference covers many control steps, so a bigger model fits in the time budget.

Temporal ensembling runs inference every step anyway and averages the overlapping predictions for the current timestep, weighting them by $w_i = e^{-m\, i}$, where $w_0$ is the oldest prediction. It smooths motion at some cost to reactivity.

The chunk length is a trade-off. Too long, and the robot runs open loop on stale information. Too short, and compounding error and jitter return. Modern VLAs use chunks of 0.5–2 seconds and replan well before the chunk ends (D12).

MechanismFrom first principles

Start with the forward process. Take a data sample $x_0$, here an action chunk, and add Gaussian noise in $K$ increasingly strong levels:

$$x_k = \sqrt{\bar\alpha_k}\,x_0 + \sqrt{1-\bar\alpha_k}\,\epsilon, \qquad \epsilon\sim\mathcal N(0, I)$$

By the last level, $\bar\alpha_K \approx 0$ and $x_K$ is pure noise. Train a network $\epsilon_\theta(x_k, k, o)$ to predict the noise that was added, conditioned on the observation $o$, with the simple loss $\lVert \epsilon - \epsilon_\theta\rVert^2$.

Sampling runs the process backwards. Start from pure noise and repeatedly subtract a little of the predicted noise. DDPM takes around a hundred small stochastic steps; the DDIM sampler takes about 10–20 deterministic ones.

Why does it handle multimodality? At high noise the predicted clean sample is a blend of all the modes. As noise falls, the blend sharpens toward whichever mode a particle is nearest. The randomness of the starting noise decides which mode, so the sampler represents all of them.

Diffusion Policy (Chi et al., 2023) applies this to action chunks. The CNN version conditions a 1D temporal U-Net on visual features through FiLM layers; a transformer version conditions through cross-attention. It observes 2 frames, predicts 16 action steps and executes 8 before replanning, a receding horizon. Training uses 100 noise levels; DDIM with about 10 steps at inference gives roughly 0.1 s latency on a desktop GPU.

MechanismFrom first principles

Draw noise $x_0\sim\mathcal N(0, I)$ and a data sample $x_1$. Connect them with a straight line:

$$x_\tau = (1-\tau)\,x_0 + \tau\, x_1, \qquad \tau\in[0,1]$$

Along that line the velocity is constant: $x_1 - x_0$. Train a network $v_\theta(x_\tau, \tau, o)$ to predict it:

$$\mathcal L = \mathbb E\,\big\lVert v_\theta(x_\tau,\tau,o) - (x_1 - x_0)\big\rVert^2$$

Averaged over all pairs, the network learns the velocity field that transports the whole noise distribution onto the data distribution. At inference, start from noise and integrate the ordinary differential equation with Euler steps: $x_{\tau+\Delta} = x_\tau + \Delta\, v_\theta(x_\tau,\tau,o)$.

Why it displaced diffusion as the default in π0, GR00T and Atlas’s large behavior model: the objective is simpler, there is no noise schedule to tune, and integration is deterministic. Because the learned paths are nearly straight, 4–10 steps suffice (π0 uses 10, π0.7 uses 5, GR00T N1 uses 4), compared with the 100 noise levels Diffusion Policy trained with. Deterministic diffusion samplers such as DDIM narrow that gap considerably, as the figure shows; the practical difference is as much about simplicity as speed. Expressiveness is essentially the same, and both remain iterative, costing a few tens of milliseconds per chunk.

MechanismFrom first principles

A vision-language model (VLM) has three parts:

1. A vision encoder, usually a ViT such as SigLIP or DINOv2 (A7), which turns an image into patch embeddings.
2. A projector, a small network that maps those embeddings into the language model’s token space.
3. A large language model (Gemma, Llama, Qwen and others) that processes image tokens and text tokens in one sequence.

PaliGemma, π0’s base, is SigLIP plus Gemma 2B, about 3B parameters. GR00T N1 used NVIDIA’s Eagle-2, later versions a Cosmos-Reason model; Gemini Robotics uses Gemini; π0.7 uses Gemma 3 4B.

Pretrained on internet-scale image-text data, a VLM already knows what a mug is, that a sponge is for wiping, roughly where things are relative to each other, and how to resolve “the one on the left”.

Token counts matter because the model must run on the robot. Each camera at 224×224 with 14-pixel patches is 256 tokens; three cameras plus an instruction is roughly 800 tokens before any action is produced, and attention cost grows with the square of that (A7).

MechanismFrom first principles

What it is. A policy built on a pretrained VLM (D6) that generates action chunks with a diffusion or flow model (D4, D5). “Generative” contrasts it with regression (D2) and with VLAs that output discrete action tokens (D8).

How it works, π0-style. Images become ViT tokens, the instruction becomes text tokens, and proprioception is projected into a state token. The VLM transformer processes them. A separate, smaller action expert (about 300M parameters in π0; 860M in π0.7) is a transformer that attends to the VLM’s internal keys and values, like a second expert in a mixture-of-transformers. It denoises the action chunk with flow matching, receiving the noisy chunk and the flow time $\tau$ as its own tokens.

π0 uses a blockwise causal attention mask with three blocks: [images + language], [robot state], [noisy actions]. Each block attends to itself and to earlier blocks, never to later ones. The VLM prefix therefore never sees the actions, and its keys and values can be computed once and reused across all denoising steps.

Training minimizes the flow-matching loss on (observation, instruction, action chunk) triples from demonstrations, co-trained with web tasks (D10). Inference encodes the observation once, runs 4–10 integration steps in the expert, and executes the chunk with real-time chunking (D12). Typical numbers: actions at 50 Hz, 50-step (one-second) chunks, 3–5B parameters in total.

Each design choice has a job. The VLM supplies semantics and language grounding. The generative head supplies multimodality and smooth chunks. Keeping the head separate keeps action modeling in continuous space and lets the backbone keep its pretrained representations (D8). Chunking fights compounding error (D3).

MechanismFrom first principles

Discrete binning (RT-1, RT-2, OpenVLA). Quantize each action dimension into 256 bins and predict the bins as tokens, one after another. It is simple, but at 50 Hz with 7–30 dimensions and one-second chunks it produces hundreds of highly correlated tokens. Autoregressive models then learn to copy the previous token rather than the underlying motion.

FAST (Physical Intelligence, 2025) compresses a chunk before tokenizing it:

1. Apply a discrete cosine transform (DCT) along time for each action dimension. Smooth trajectories concentrate their energy in a few low-frequency coefficients.
2. Quantize the coefficients; most small ones round to zero.
3. Byte-pair encode the result, merging frequent patterns into single tokens.

The result is far fewer tokens, which makes autoregressive VLAs viable at 50 Hz and trains several times faster than binning.

Flow and diffusion heads keep actions continuous and multimodal (D5); they are the 2026 default.

Knowledge insulation (π0.5 KI, 2025) combines the two. The VLM backbone is trained with cross-entropy on FAST tokens for actions and on language, while a flow action expert reads the backbone’s representations with gradients stopped from flowing back. Early in training, gradients from a randomly initialized continuous-action head are mostly noise; letting them into the backbone wrecks its pretrained representations, slows learning and degrades instruction following. With insulation, training is faster and language grounding survives.

MechanismFrom first principles

What. One model is trained on data from single arms, bimanual rigs, mobile manipulators and humanoids at once.

How. A shared backbone learns what is common. Small per-embodiment state encoders and action decoders handle what isn’t (GR00T uses a small MLP per embodiment). Actions are padded to a fixed maximum dimension, with the unused entries masked (π0). An embodiment tag or control-mode token tells the model which body it is driving.

Why it helps a humanoid. Object semantics, affordances and task structure are mostly independent of the body, so a data-poor humanoid can borrow them. Gemini Robotics 1.5 found that for the Apollo humanoid, external robot data provided “the greatest performance boost”, and its Motion Transfer recipe (mechanism not disclosed) aligned embodiments well enough that transfer helped even data-rich robots. π0.7 transferred shirt folding to a bimanual UR5e that had never seen the task, at 80% success.

Risks. Negative transfer between very different bodies, capacity diluted across embodiments, and sensitivity to the mixture weights.

MechanismFrom first principles

Mix robot action data with web-style tasks: visual question answering, captioning, object detection and pointing, and subtask prediction (given an image and a goal, name the next step). The same backbone is trained on all of them in one run, each with its own loss.

The reason is catastrophic forgetting. Fine-tuning a VLM only on robot actions quickly erodes its language and visual knowledge, because nothing in the robot data rewards keeping it. Co-training keeps the old objectives alive. RT-2 co-fine-tuned with web data from the start. π0.5 found co-training was what unlocked generalization to new homes. π0.7 adds egocentric human video and subgoal-image generation to the mix, and the knowledge-insulation recipe (D8) co-trains on FAST tokens and language while protecting the backbone from the action head.

MechanismFrom first principles

Post-training means collecting demonstrations for the target robot and task, then fine-tuning. The amounts are small compared with pretraining: Gemini Robotics On-Device adapts with “as few as 50 to 100” demonstrations; NVIDIA’s GR00T workflow collects about 400 teleoperated trajectories; DreamZero adapts to a new embodiment from about 30 minutes of play data; Generalist’s GEN-1 reports reaching high reliability with about an hour of robot data per task.

What changes is chosen carefully. The large language backbone is often frozen or adapted with LoRA, while the action expert and embodiment adapters are trained fully. Freezing matters because small datasets overfit, and full fine-tuning erodes the general knowledge you paid for (D10).

A newer direction skips gradient updates altogether. Generalist’s GEN-1.5 (2026) is shown a demo a few seconds long in its context and performs the task, with no fine-tuning.

MechanismFrom first principles

A 3–5B-parameter VLA takes 50–250 ms to produce a chunk. Executed naively, the robot runs a chunk, stops, waits for the next and moves in jerky bursts.

Asynchronous execution starts computing the next chunk while the current one runs. But the new chunk was computed from an observation that is now stale, and because the policy is multimodal (D2) it may have chosen a different way to do the same thing. Its first actions can disagree with what the robot is doing at that moment, producing a jump at the seam.

Real-time chunking (RTC, Physical Intelligence, 2025) treats the actions that will be executed during the inference delay as already committed. It generates the new chunk as an inpainting problem: during denoising, the overlapping prefix is guided to match the executing actions, with the constraint fading over the rest of the overlap. The new chunk continues the current motion smoothly and still reacts to the new observation.

Two related tricks. π0.7 builds latency into training by simulating up to 240 ms of delay, so the model expects its outputs to take effect late. Boston Dynamics’ Atlas behavior model predicts each action with a timestamp, so execution can be sped up 1.5–2× at inference without retraining.

MechanismFrom first principles

System 2 and System 1. The names borrow Kahneman’s fast/slow framing. Figure’s Helix runs a 7B VLM (“S2”) at 7–9 Hz that emits a latent vector, and an 80M visuomotor policy (“S1”) at 200 Hz that consumes the latest latent plus fresh camera and proprioceptive input and outputs body commands. Helix 02 added S0, the 1 kHz whole-body controller of C6. GR00T N1 similarly runs its VLM at about 10 Hz and its diffusion-transformer action head at 120 Hz.

Why split. A multi-billion-parameter model can’t close a 200 Hz loop, but dexterity needs high-rate visual feedback, for instance to catch a slipping object. The split also lets each part scale independently and train on different data. The costs: the latent passed from S2 to S1 is an information bottleneck, S1 can’t reason, the two must stay synchronized, and failures are harder to attribute.

Embodied-reasoning orchestrators. Gemini Robotics-ER 1.5 is a VLM fine-tuned for spatial reasoning, pointing, success detection and planning. It sits above the VLA, calls it with natural-language subtasks, monitors progress, replans, and can use tools such as web search. On long-horizon tasks the combined agent “frequently achieves scores near 80%”, against “up to 44 percent” for the thinking VLA alone (task-progress scores). Physical Intelligence’s Hi Robot (2025) and MEM (2026) explore the same space: high-level reasoning over subtasks, and memory for tasks lasting up to about 15 minutes.

The single-stream alternative. Generalist’s GEN-0 rejects the split with Harmonic Reasoning: one model interleaves asynchronous “thinking” and action tokens, so reasoning proceeds while actions stream. Whether that beats an explicit hierarchy at scale is an open question.

MechanismFrom first principles

Imitation learning can reach 50–80% success on hard tasks. It then stalls, for two structural reasons:

1. It can’t surpass the demonstrations. BC reproduces the demonstrators’ behavior, hesitations and slow moves included.
2. It never sees its own failures. The data contains the expert’s states, not the states the policy drifts into (D1), so it can’t learn how to recover.

Deployment needs something else entirely: 99% or better. The arithmetic of multi-step tasks shows why. If each of $n$ steps succeeds independently with probability $p$, the whole task succeeds with probability $p^{n}$. A policy that gets each of 10 subtasks right 95% of the time finishes the task only 60% of the time.

Physical Intelligence put the problem plainly: “It’s not hard to get a robot to succeed half of the time, but it’s very hard to get it to succeed every time.” RL on the robot’s own rollouts, labeled good or bad, targets exactly the states imitation never covered.

MechanismFrom first principles

Record autonomous episodes with outcome labels: success or failure, time to completion, or a human rating. Train a value function $V(o)$ to predict the expected outcome from an observation, by regression on those labels. A common choice for manipulation is the negative number of steps remaining until success, with a large penalty for failure.

For each step, estimate the advantage of the action actually taken: how much better the realized outcome was than $V$ predicted. A one-step version is the temporal-difference error of B2:

$$A_t \approx r_t + V(o_{t+1}) - V(o_t)$$

With $r_t = -1$ per step, an action whose next state is predicted to be much closer to success than the current one gets a positive advantage; a fumble that pushes success further away gets a negative one.

This turns a single sparse label, success at the end, into a dense signal about which moments mattered. It works even on failed episodes: the steps before the fumble may have been fine, and the recovery after it may have been excellent.

MechanismFrom first principles

Recap, from Physical Intelligence’s π*0.6 (2025), turns RL into supervised learning in five steps:

1. Train a value function on labeled episodes (E2).
2. Compute each action’s advantage.
3. Binarize it into an improvement indicator: “good” if the advantage exceeds a threshold, otherwise “not good”.
4. Train the VLA on all the data, demonstrations, autonomous rollouts and teleoperated corrections alike, with the indicator as an extra input token.
5. At test time, always condition on “good”.

The policy learns what distinguishes better actions from worse ones in the same situation, and is then asked for the better kind.

Why not PPO? Policy-gradient methods need $\log\pi(a\mid o)$, which flow and diffusion policies can’t compute tractably. They need on-policy samples at scale, which are ruinously expensive in the real world. And they throw data away after each update (B2). Advantage conditioning reuses every episode offline, trains as stably as supervised learning, and handles a pipeline of offline pretraining, then demonstrations, then on-robot rollouts with corrections. On the hardest tasks, Physical Intelligence reports that it “more than doubles task throughput and roughly halves the task failure rate”.

MechanismFrom first principles

Interventions. An operator watches the autonomous robot and takes control when it is about to fail. Each intervention produces two kinds of data. The intervention segment is a corrective demonstration, recorded in exactly the states the policy gets into. The autonomous segment leading up to it can be labeled as bad. This is the cheapest way to collect the data that compounding error calls for (D1); it is DAgger with a human as the expert (B7).

HIL-SERL (Luo et al., 2024) does real-world RL with four ingredients:

• an off-policy, sample-efficient RL algorithm (SAC-style, B2);
• RLPD: every training batch mixes demonstrations and online experience at a fixed ratio, typically half and half;
• a learned reward classifier that judges success from images;
• human interventions, recorded as data.

It reached near-perfect success on precise insertion and assembly tasks within 1–2.5 hours of real training.

Why off-policy here, when PPO dominates simulation? Because real samples are precious. Reusing every transition many times matters more than PPO’s stability at huge scale.

MechanismFrom first principles

Metadata conditioning. π0.7 tags each episode, and each segment within it, with metadata: quality, speed, whether mistakes occurred, and the control mode used. The model is trained conditioned on those tags, with the tags randomly dropped some of the time so that it also learns the unconditional distribution. Without metadata, adding lower-quality data hurt; with it, performance kept improving as data grew, because the model can separate “what happened” from “what good looks like”.

Classifier-free guidance (CFG) then amplifies the conditioning at inference. A flow or diffusion model trained this way can produce two predictions for the same input, one conditioned on “high quality, no mistakes” and one unconditioned. Extrapolate from the second toward the first:

$$v = v(x,\varnothing) + \beta\,\big(v(x,c) - v(x,\varnothing)\big)$$

With $\beta = 1$ this is ordinary conditioning; $\beta > 1$ pushes samples further from whatever the unconditional model would do and toward what distinguishes the condition. π0.7 used guidance weights of 1.3, 1.7 and 2.2.

MechanismFrom first principles

A specialist is a copy of the generalist fine-tuned with RL (E3, E4) until it is excellent at one task. Keeping one model per task doesn’t scale, and specialists forget how to do everything else.

So distill. Run each specialist, collect its rollouts, tag them with metadata (E5), and add them to the generalist’s training mixture. π0.7 ingested the rollouts of π*0.6 specialists this way and matched or beat them “out of the box” on their own tasks. Ablations that removed the evaluation rollouts or the metadata underperformed.

This is the teacher-student idea of B7 applied at the level of whole foundation models: the teacher is a privileged specialist, and the student is the model you ship.

MechanismFrom first principles

A latent action model has two parts. An encoder looks at a pair of frames $(x_t, x_{t+1})$ and outputs a code from a small, discrete codebook (vector quantization, as in a VQ-VAE). A decoder must reconstruct $x_{t+1}$ from $x_t$ and that code alone. Because the code is the only extra information the decoder gets, it is forced to capture what changed, which in a video of a robot or a hand is mostly the action, and it learns this without any labels.

The recipe then has three steps:

1. Train the latent action model on lots of video.
2. Label every video with latent codes, and pretrain a policy to predict those codes from images and instructions.
3. Learn a small head that maps latent codes to real robot actions, from a little robot data.

The camera-shake failure is real. Latents capture whatever change dominates the pixels. That can be camera motion or lighting rather than intent, and they capture coarse motion rather than force or precision.

GR00T N1 treated latent-labeled video as its own “LAPA embodiment” in the pretraining mix (D9); AgiBot’s GO-1 puts a latent planner between its VLM and its action expert.

MechanismFrom first principles

An inverse dynamics model (IDM) is a function $f(o_t, o_{t+1}) \to a_t$. It is trained by supervised learning on robot data, where actions are known, and then used to pseudo-label videos that have no actions.

Inverse dynamics is easier than policy learning: given the next frame, the action is mostly determined, with no need to decide what to do. That is why OpenAI’s VPT (2022) could train an IDM on a small labeled set and use it to label 70,000 hours of Minecraft video for behavior cloning.

The catch is domain. An IDM is accurate only on video that looks like its training data. Its main use is therefore labeling generated robot videos, which do look like the robot (F3), rather than human videos, which don’t. In GR00T N1’s experiments, latent-action labels were slightly better with 30 demonstrations per task, and IDM labels pulled ahead with 100 and 300.

MechanismFrom first principles

A video world model predicts future frames given past frames and a text prompt or an action. NVIDIA’s Cosmos, the 1X World Model and Google DeepMind’s Genie 3 (real-time, interactive worlds) are examples.

DreamGen (NVIDIA’s GR00T-Dreams) turns one into a data engine:

1. Fine-tune a video model on the target robot’s own videos.
2. Prompt it with new tasks and new scenes.
3. Generate videos of the robot doing them.
4. Pseudo-label the videos with an IDM (F2) or latent actions (F1).
5. Train the policy on these neural trajectories.

The results are real but modest. GR00T N1 expanded 88 hours of teleoperation into 827 hours of neural trajectories, using about 105,000 L40 GPU-hours (roughly a day and a half on 3,600 GPUs), and gained 4–9 points on the RoboCasa benchmark. DreamGen taught a humanoid 22 new behaviors, with 43% success in seen environments and 28.5% in unseen ones. NVIDIA’s framing that this replaces three months of collection with 36 hours comes from a press release.

Read it as a multiplier in low-data regimes, not a replacement for real data.

MechanismFrom first principles

Joint-embedding predictive architectures (JEPAs) encode inputs into a latent space and train a predictor to predict the latent representation of masked or future parts of the input, rather than their pixels. The loss is computed between embeddings, so the model spends no capacity on details that don’t matter for understanding: exact textures, lighting flicker, sensor noise.

Meta’s V-JEPA 2 (2025) was pretrained this way on over a million hours of internet video. V-JEPA 2-AC then added an action-conditioned predictor, trained on under 62 hours of robot video from the DROID dataset: given the current embedding and an action, predict the next embedding.

Planning then works as follows. Encode the current image and a goal image. Sample many candidate action sequences, roll each forward with the predictor, and score it by how close the final predicted embedding is to the goal’s. Keep the best, resample around them, repeat (the cross-entropy method), and execute the first action. With this, V-JEPA 2-AC performed zero-shot pick-and-place on Franka arms in labs it had never seen.

The trade-offs: planning in latent space is slow (many predictor calls per action) and coarse, so precision is limited.

MechanismFrom first principles

What. A world-action model (WAM) generates both a prediction of the future (video frames or latent states) and the action chunk, in one model. Predicting the world is both a pretraining objective and a source of understanding.

How (DreamZero, NVIDIA, 2026). Start from a 14-billion-parameter video-diffusion model (Wan). Fine-tune it to denoise future video frames and the action chunk together, conditioned on the current frames and a language instruction. Run it closed loop at about 7 Hz. In the authors’ words, action learning shifts “from dense state–action imitation to inverse dynamics”: the model first imagines what should happen, and the actions follow.

Why. Video pretraining on the internet brings physics, object permanence and scene understanding that no robot dataset can, and generating the future is a rich auxiliary signal. DreamZero reports more than 2× better generalization to new tasks and environments than state-of-the-art VLAs, a 42% relative gain from 10–20 minutes of video-only demonstrations, and adaptation to a new robot from 30 minutes of play data. NVIDIA previewed GR00T N2 as built on the same architecture.

Trade-offs. Far more compute (14B parameters of video diffusion) and a 7 Hz closed loop, versus 50–200 Hz for visuomotor policies. It is unproven on contact-rich precision and whole-body dexterity, and a video backbone brings weaker language reasoning than an LLM backbone. Compared with using a world model only to generate data (F3), joint training avoids the chain of pseudo-labeling errors.

MechanismFrom first principles

Physical evaluation is the bottleneck of iteration. 1X put it this way: “physically evaluating each policy we’ve trained across these varied scenarios would take several lifetimes.”

The 1X World Model (2025) predicts future observations and task success from a starting frame and a candidate policy’s actions, so checkpoints can be ranked without running the robot. NVIDIA’s Cosmos and Isaac Lab-Arena serve similar roles in simulation, and Google DeepMind has evaluated Gemini Robotics policies inside its Veo video model, including safety scenarios.

The value is in ranking, not absolute numbers. A world model can be biased, say optimistic across the board, and still order policies correctly. What breaks it is error that differs between policies, for instance hallucinating success for exactly the behaviors a new policy exploits.

MechanismFrom first principles

MimicGen’s insight is that most manipulation demos decompose into object-centric subtasks: approach the mug, grasp it, carry it, place it. Within a subtask, the motion is naturally expressed relative to the relevant object’s pose.

So:

1. Segment each human demo into object-centric subtasks.
2. Express each segment’s end-effector poses in the frame of the relevant object.
3. In a new simulated scene with objects elsewhere, transform each segment to the new object pose, and connect segments with interpolated motion.
4. Execute in simulation and keep only the trajectories that succeed.

MimicGen generated over 50,000 simulated demonstrations across 18 tasks from about 200 human ones; DexMimicGen extended it to bimanual dexterous humanoids. GR00T N1 generated 780,000 simulated trajectories, the equivalent of about 6,500 hours, in 11 hours.

The limits are clear. It produces only variations of skills that were demonstrated, and the data is simulated, so it needs visual randomization or co-training with real data to help on hardware (D10).

MechanismFrom first principles

An LLM’s training data is text, and text is also what it produces, so the internet is directly usable training data. A robot’s output is actions at 30–1,000 Hz across 20–50 degrees of freedom. Video shows what happened, not which motor commands caused it (F1, F2).

Five things compound the problem:

• The embodiment gap. Data from one robot is only partly useful for another with different kinematics, sensors and dynamics (D9).
• Invisible physics. Contact forces, friction and grip pressure don’t appear in video at all (G8).
• Long, multimodal horizons. A single task can be done many valid ways over minutes (D2).
• Real-time collection. Teleoperation produces data only as fast as a person can work, one robot at a time.
• Diversity is expensive. The variable that matters most for generalization, the number of distinct environments, is the hardest to scale with robots (G7).

MechanismFrom first principles

The main ways to teleoperate:

Why teleoperation data is gold: it is recorded on the robot’s own body, cameras and controller, so there is no embodiment gap. Why it is hard to learn from: operators are multimodal and non-Markovian (D2, D3); they hesitate and correct; and without haptics, grasps are often sloppy.

Teleoperate through the controller you will deploy. NVIDIA’s GR00T workflow, for example, teleoperates through the whole-body controller that will later run under the policy, and collects about 400 demonstrations. The controller is the mapping from commands to motion; the policy learns to emit commands; swap the controller and every learned command means something slightly different.

MechanismFrom first principles

Egocentric means first-person: a head-mounted camera plus estimated hand (and sometimes body) poses. Devices include Apple Vision Pro (the EgoDex dataset: 829 hours with 3D hand and finger tracking), Meta’s Project Aria glasses, Figure’s Index program (an app plus a Figure-supplied recording device), the five-camera helmet-and-backpack rigs Tesla reportedly adopted in 2025, and Generalist’s devices.

Three properties make it attractive. The viewpoint matches: a humanoid’s head cameras sit roughly where a person’s eyes are. Head motion encodes attention. And collection happens in real homes and workplaces, giving environment diversity, the variable that matters most for generalization (G7), at a small fraction of the cost of teleoperation.

How it becomes usable. Treat “human” as one more embodiment whose actions are hand-pose trajectories (position, orientation, finger state), and co-train it in the cross-embodiment mixture (D9), or pretrain on it alone. The model learns to map scenes and instructions to hand trajectories; fine-tuning on robot data then learns the hand-to-gripper correspondence and the robot’s own dynamics.

What “measurably transfers” means. Pretraining on human data improves the robot’s success on robot tasks, in controlled comparisons:

• Figure Helix 2.5 (September 2026). The same policy, trained from scratch or first pretrained (from random initialization) on Index human video, was evaluated blind and zero-shot on full tasks (tidying, towel folding, bed making) in 30 homes it had never seen: 9% versus 56% success. On one task it matched a Helix 02 policy while using half as much adaptation data.
• Physical Intelligence (December 2025). Human data does nothing until the robot pretraining set is large and diverse enough for the model to align the two; past that point, fine-tuning with human video nearly doubles performance on tasks shown only in the human videos.
• EgoMimic (2024): an additional hour of hand data was worth more than an additional hour of robot data. GR00T N1.5’s FLARE objective lifted novel-object success from 15% to 55% using human videos. Figure’s Project Go-Big learned navigation from egocentric video with no robot demonstrations at all.

Measured scaling curves. Train a series of models on increasing amounts of human data (Figure used four over an 8× range), record validation loss on held-out data, and fit a scaling law, typically a power law $L(D) = (D_c/D)^{\alpha}$, which is a straight line on log-log axes. Figure reports that its fit predicted the largest run’s test loss “to four decimal places” before training began (N2). That makes human data a resource you can budget against, the way LLM labs budget text tokens.

Why it isn’t sufficient. Human video contains no actions in the robot’s space and no contact or force information; hand-pose estimation is noisy; people use strategies the robot’s hand can’t execute; and a residual embodiment gap must be bridged with robot data. All three headline results above are company-reported, not peer-reviewed.

MechanismFrom first principles

UMI (Chi et al., 2024) is a 3D-printed handheld gripper carrying a GoPro with a wide fisheye lens at the wrist. Its bill of materials is about $73 for the gripper and $298 for the camera and accessories, about $371 in all. The camera’s visual-inertial tracking recovers the gripper’s 6-DoF pose over time, and the gripper width is read from the image.

A person carries it into any kitchen or shop and demonstrates. At deployment, the robot holds an identical gripper with the same camera. The policy maps wrist-camera images to end-effector pose trajectories and gripper widths, so the robot sees exactly what the demonstrator’s camera saw.

The trade-offs: no proprioception (only the gripper’s pose), actions tied to that specific gripper, and latency differences between recording and deployment that must be measured and matched.

MechanismFrom first principles

The pyramid. NVIDIA’s GR00T N1 framed its training data as a pyramid, and the shape is general:

• Bottom: web and human video. Nearly free, huge, no robot actions. It supplies semantics and physical priors (D6, G3).
• Middle: simulation and neural trajectories. Scales with compute and has a reality gap. It supplies coverage of variations (F3, F7).
• Top: real robot data. Scarce, expensive, exactly on-distribution. It grounds the model in the real body and real dynamics (G2).

The shape is a cost-versus-relevance trade-off: the closer data is to the robot, the more it costs per hour.

The flywheel turns a fixed pyramid into a growing one. Deployment produces rollouts and interventions on exactly the hardest states. RL post-training converts them into improvement (Part E). Better models deploy more widely and produce more data. 1X’s home robot, for example, launched with remote teleoperation by 1X experts for tasks it can’t yet do alone.

Helix 2.5 reshaped the pyramid: it was pretrained from scratch on human video, with no web-pretrained VLM underneath (D6), so human video served as both the base and the motor prior.

MechanismFrom first principles

Kaplan et al. (2020) found that language-model loss falls as a power law in parameters, data and compute over many orders of magnitude; Chinchilla (2022) showed how to split a compute budget between model size and data. A power law $L = (D_c/D)^{\alpha}$ is a straight line on log-log axes, so a few small runs can predict a large one.

Evidence in robotics so far:

• Figure’s human-data curve (2026): four runs over an 8× data range predicted the largest run’s test loss to four decimal places.
• Generalist’s GEN-0 (2025): post-training loss falls with pretraining data, and there is a phase transition around 7B parameters. Models of 1B “ossify”: they stop absorbing more data.
• NVIDIA’s SONIC (2025): motion-tracking quality kept improving as data grew from 4M to 100M frames, compute from about 2,000 to 21,000 GPU-hours, and the model from 1.2M to 42M parameters (M10).

The caveat. These laws are measured mostly on validation action-prediction loss, which is only loosely coupled to closed-loop success: a chunk with low mean squared error can still miss a grasp. They are measured within one organization’s data distribution, and rarely replicated. Treat them as budgeting tools for training, not as predictions of task success (H1).

MechanismFrom first principles

Lin et al. (Tsinghua, ICLR 2025) ran the most thorough controlled study: over 40,000 demonstrations and more than 15,000 real-world rollouts. Their findings:

• Generalization to new environments and objects follows a roughly power-law relationship with the number of environments and objects in training.
• Once the number of demonstrations per environment or object passes a threshold, additional demonstrations have minimal effect.
• A practical recipe: about 32 environments, each with a unique object, with 50 demonstrations each, reached around 90% success on novel environments and objects for the tasks studied.

π0.7’s ablations agree: dropping the most task-diverse 20% of the data hurt far more than dropping a random 20%.

This is why cheap, diverse human data (G3) matters so much, and why repetitive teleoperation of the same task in the same lab saturates quickly.

MechanismFrom first principles

Tactile sensors come in several families:

• Camera-based (GelSight and similar). A soft elastomer pad with a reflective skin is imaged from behind by a small camera, lit by colored lights from different directions. Where the pad is pressed, its surface tilts, and each color channel brightens or darkens depending on which way it tilts. This is photometric stereo: shape is recovered from shading. Printed markers track shear and slip.
• Capacitive or magnetic skins. Arrays of small sensing elements; thinner, lower resolution.
• Force-torque sensors at the wrist or fingertip: a single 6-axis measurement.

Figure 03’s fingertips register forces as small as 3 grams, and Helix 02 feeds tactile and palm-camera input directly into its visuomotor policy S1 (D13).

The data problem. There is no standard tactile sensor or data format, so tactile data doesn’t benefit from cross-embodiment pooling (D9), and human video contains no touch at all (G3).

Dexterous hands add many degrees of freedom (Figure 02’s hands have 16; Tesla reported 22 for Optimus’s third-generation hand), make retargeting lossy (C2), and are hard to teleoperate without haptics. In-hand reorientation is therefore still mostly learned in simulation: OpenAI’s Rubik’s cube hand with automatic domain randomization, NVIDIA’s DexPBT, and Berkeley’s proprioception-only in-hand rotation using RMA (B6).

MechanismFrom first principles

Each trial either succeeds or fails. If the policy’s true success rate is $p$, then $k$ successes in $n$ trials give the estimate $\hat p = k/n$, whose standard error is

$$\mathrm{SE}(\hat p) = \sqrt{\frac{p\,(1-p)}{n}}$$

At $p = 0.5$ and $n = 20$, that is 11 percentage points: a 95% interval spans roughly ±22 points. Running 20 trials and reporting “55%” says the true rate is probably somewhere between about 34% and 74%.

Which interval. The textbook interval $\hat p \pm 1.96\,\mathrm{SE}$ misbehaves exactly where robotics lives: few trials and rates near 0% or 100% (after 10 successes out of 10, it claims zero uncertainty). The Wilson score interval fixes this and is the sensible default:

$$\frac{\hat p + \frac{z^2}{2n}}{1+\frac{z^2}{n}} \;\pm\; \frac{z}{1+\frac{z^2}{n}}\sqrt{\frac{\hat p(1-\hat p)}{n} + \frac{z^2}{4n^2}}, \qquad z = 1.96$$

After 10 out of 10, Wilson still admits the true rate could be as low as 72%.

Comparing two policies is harder than measuring one, because both estimates are noisy. To detect an improvement from 60% to 70% with the conventional 80% chance of noticing it (statistical power), you need about 356 trials per policy. Most robot papers run 10–50.

Good evaluation design, which matters as much as the number of trials:

• Blind. The person resetting and scoring doesn’t know which policy is running.
• Randomized and interleaved. Alternate policies in random order, so drifts in lighting, battery charge or wear don’t favor one.
• Paired. Run both policies from matched initial conditions; this cancels much of the variation between scenes.
• Fixed in advance. Decide the number of trials before looking at results; stopping when the numbers look good inflates false positives.
• Graded. Partial-credit progress scores and per-step success (E1) carry more information per trial than a single pass/fail.

Toyota Research Institute’s 2025 study of large behavior models ran blind, randomized trials and warned of a “significant risk that many robotics papers are measuring statistical noise due to insufficient statistical power.” RoboArena (2025) pools evaluations across institutions: evaluators see two anonymous policies on the same task and say which did better, and the pairwise outcomes are combined into a ranking, as Chatbot Arena does for language models.

MechanismFrom first principles

A benchmark specifies what counts as success, which initial states are drawn and how, what data (if any) may be used for training, and how results are reported. Four kinds are in use:

• Simulation suites. LIBERO (130 tasks in four suites), RoboCasa (kitchens with 100 tasks and more than 150 object categories), BEHAVIOR-1K (1,000 household activities across 50 scenes), ManiSkill3 (GPU-parallel simulation at more than 30,000 frames per second), HumanoidBench (whole-body humanoid tasks with dexterous hands) and Isaac Lab-Arena (composable evaluation for generalist policies).
• Real-to-sim replicas. SimplerEnv rebuilds real lab setups in simulation, tuned so that policy rankings in simulation track rankings on the real robots.
• Distributed real-world evaluation. RoboArena runs double-blind pairwise comparisons on identical robot platforms at several institutions.
• Learned simulators. World models that predict outcomes of rollouts (F6).

Saturation and overfitting. A benchmark stops being informative when the best methods max it out. Leading VLAs now average above 95% on LIBERO (OpenVLA-OFT reported 97.1% across its four suites). LIBERO-Plus (2025) re-tested such models under modest perturbations of camera, lighting, robot initial state and language, and found scores of 95% could drop below 30%. This is Goodhart’s law: once the benchmark is the target, methods fit its specifics. Selection alone inflates scores, too: try many variants, report the best, and the reported number is biased upward even if no variant is really better (the winner’s curse, in the second tab).

The humanoid gap. There is no widely shared benchmark for whole-body humanoid behavior on real hardware. Humanoid results are mostly company-reported demonstrations and internal evaluations, which is one reason blind protocols (H1) matter.