Robotics & Embodied AI Engineer

The world does not wait for the model

A language model's whole world is the prompt. It can take as long as it likes, nothing changes while it thinks, and a wrong answer costs a retry. A robot lives in a loop: it senses, it decides, it acts, and the world has already moved by the time the next sensor frame arrives. Everything that makes embodied AI hard follows from that loop. Four properties of the loop deserve names, because each one breaks a different assumption that software engineers carry over from the screen.

Partial observability

A robot never sees its state. It sees measurements: a camera frame, a joint encoder reading, a force sensor value. The cup behind the kettle is still there, but not in the image. The encoder says the joint is at 31.2 degrees, but the gearbox has backlash and the link is actually at 31.5. In the formal language of the field, the robot acts in a partially observable Markov decision process: it must carry a belief about the state, not the state itself, and the belief is only as good as the history of measurements that produced it. Practically, this means a policy must either remember (keep an internal state) or estimate (run a filter, see module 3), and it must be written so that a missing or stale measurement degrades behaviour gracefully rather than catastrophically.

Latency

Every stage of the loop adds delay: camera exposure and readout, image transfer, inference, command transmission, motor controller response, and finally the physical inertia of the arm. These are additive. A policy that takes 80 ms to run on a GPU is not an 80 ms policy once the rest of the chain is added; 150–250 ms end-to-end is a normal figure for a vision-based controller. At 1 m/s of end-effector speed, 200 ms is 20 cm of travel on information that is already old.

Delay does something worse than make you late: it makes feedback loops unstable. A controller that corrects an error it measured 200 ms ago is correcting a position the robot no longer occupies. Push the gain up to react faster and the system overshoots, then over-corrects the overshoot, and oscillates. This is exactly what the mission for this module asks you to see with your own eyes: the same proportional controller, with 0, 2 and 6 steps of sensor delay, goes from a clean approach to a wobble to a growing oscillation.

Physics

Contact is the heart of manipulation and the thing simulators and models handle worst. Friction switches between sticking and slipping with no warning. A grasp that is 2 mm off closes on air. Pushing an object across a table is a sequence of tiny contact events that depend on surface properties nobody measured. And a physical error is often irreversible: you cannot un-drop the glass. Russ Tedrake's Robotic Manipulation opens by arguing that this is why manipulation, not locomotion or navigation, remains the frontier: the geometry is easy, the contact dynamics are not.

Safety

The cost of errors is asymmetric. A thousand correct grasps do not pay for one that pins a hand to a table. This changes how you evaluate: average return is the wrong metric when the tail matters, and a policy with 98% success and 2% dangerous failure is worse than one with 90% success and 10% harmless failure. It also changes architecture: a learned policy is a proposal, and something deterministic, simple and verifiable has the final say on the motors. Module 7 builds that layer.

A worked example: the delayed proportional controller

Take a point robot on a line at position x, target at 0, running at 50 Hz (dt = 0.02 s). Each step it reads a position measurement z = x + noise, and commands velocity v = -k · z, so x moves by v · dt. With k = 10 and no delay, x halves every few steps and settles. Now make the measurement arrive d steps late, so the command uses z from d steps ago. With d = 2 the response overshoots and rings. With d = 6 the ring grows, because by the time the controller sees the robot pass the target it has already commanded it several steps further. The fix is not a smarter model: it is a lower gain, a delay-aware estimate of where the robot is now, or a shorter loop. Nothing here involves learning, yet the same thing happens to a neural policy, which is why robot learning engineers need the control literature as much as the ML literature.

Why a good benchmark score can still be unusable

Three things that benchmarks usually ignore: timing (did the simulator pause while the policy thought?), distribution shift (the real table is a different colour and the light comes from a window), and rare failures (50 episodes cannot see a 1-in-500 catastrophic mode). Rodney Brooks's 1991 argument in Intelligence Without Representation, that intelligence should be tested in the world rather than in a model of the world, still reads as a warning against trusting the simulator too much. Sergey Levine's Understanding the World Through Action makes the compatible case that the richest learning signal is interaction, which is precisely why robots are hard to train from static datasets alone.

What is still unsettled: how much of the loop should be learned end-to-end and how much should stay as engineered estimation and control. The practice this track follows is "learn what you cannot write by hand, engineer what you must be able to trust".

What to remember

• A robot acts on measurements, not state; design for stale and missing data.
• Latency is additive along the whole chain and turns healthy feedback into oscillation.
• Contact physics is discontinuous and often irreversible; simulation is an approximation of it.
• Safety is asymmetric, so tail failures matter more than mean performance.
• Learned policies propose; a simple verifiable layer disposes.