With the twirl of a thumbstick, squeeze of a trigger, and press of a few buttons, even an unskilled player can dance their way through a 3D video game environment. A British startup is wagering that straightforward sequences like these contain a trove of information that can be used to train new artificial intelligence models.
\n\nThere is a growing belief in corners of the AI industry that large language models will ultimately be limited by their inability to navigate the physical world. Trained only on words, LLMs are perhaps ill-equipped to pilot autonomous vehicles, steer robotic arms, or perform any other action that requires finesse and precision. To remedy that shortcoming, a crop of celebrated researchers including Fei-Fei Li and Yann LeCun are focusing their efforts on a different class of AI: world models.
\n\nTo become fluent in real-world physics, world models need to be trained on a combination of visual and action data. Before it can deftly maneuver a robotic arm, a model might need to be trained on video footage of the factory floor, coupled with information about how firmly an object should be gripped, with what torque it’s manipulated, and so on. But unlike LLMs, trained on…
Original source: https://www.wired.com/