Where does the simulator come from?
The prerequisite AlphaZero course ends on a cliffhanger. Self-play worked because Go gave us a perfect simulator for free: the rules of the game. Feed the search a position, it hands back the next one, and the flywheel spins. But the real world does not ship with rules. A robot reaching for a cup has no env.step() to call.
Four landmark systems give four escalating answers, and you will rebuild each one small enough to read in a sitting. MuZero learns the game's transitions and plans inside them. Dreamer learns a simulator from pixels and trains its policy inside the dream. JEPA predicts in representation space and skips pixels entirely. Genie learns the simulator and its action space from raw video. Animated equations, an in-browser MuJoCo robot arm, real-home robot data, labs, and flashcards throughout.
Start module 1: MuZero ›Sort environments by what you know about their rules. Every module is a different answer to losing the rulebook, and each one lives at a different point on the axis.
AlphaZero's search calls the real rules at every node. Delete that call and train three small networks to do the job: h encodes the board, g predicts the next state, f judges positions. Then measure exactly where the learned model is right and where it drifts.
Learn a latent simulator from pixels and train the policy entirely on imagined rollouts. Features the in-browser MuJoCo robot arm: the same engine, the same scene, running in your tab.
Skip the pixels entirely: predict what the next moment means, not what it looks like. Along the way, meet collapse, the failure mode every model trained this way has to avoid.
No action labels, just raw video: discover a latent action space from footage alone, and turn a video model into a playable world.
The same loop that mastered Go, predict, search, distill, resurfaces in reasoning LLMs. What actually transfers, and what is marketing.
An any-to-any model that predicts its own next observation is also a world model. This satellite ties the course to the nano-omni track.
The world keeps changing after training ends. What it takes to keep a learned simulator current, and why that is still an open problem.
Pick an environment, pick what your model should predict, and defend the choice with measurements.