Some elite AI researchers say language is limiting. Here's the new kind of model they are building instead.

Top AI researchers like Fei-Fei Li and Yann LeCun are thinking about AI beyond LLMs. At World Labs, Li is focused on building world models. LeCun is building them at his new startup. World models mimic the mental constructs that humans create in their minds. As OpenAI, Anthropic, and Big Tech invest billions in developing state-of-the-art large language models, a small group of elite AI researchers is working on what they say is the next big thing. Computer scientists like Fei-Fei Li , the Stanford professor famous for inventing ImageNet, and Yann LeCun , Meta's outgoing chief AI scientist, are building what they call "world models." Unlike large language models , which determine outputs based on statistical relationships between words and phrases, world models anticipate outcomes by mimicking the mental constructs that humans make of the world around them. "Humans," Li said on an episode of Andreessen Horowitz's A16z podcast in June, "not only do we survive, live, and work, but we build civilization beyond language." Put simply, world models are AI systems that anticipate what will happen next, much like humans use intuition based on their experience to predict the consequences of their actions. Think of a child who, with no language skills, will learn to understand that if they push a toy car, it will roll. Computer scientist and MIT professor, Jay Wright Forrester, in his 1971 paper "Counterintuitive Behavior of Social Systems," explained why mental models are crucial to human behavior: Each of us uses models constantly. Every person in private life and in business instinctively uses models for decision making. The mental images in one's head about one's surroundings are models. One's head does not contain real families, businesses, cities, governments, or countries. One uses selected concepts and relationships to represent real systems. A mental image is a model. All decisions are taken on the basis of models. All laws are passed on the basis of models. All executive actions are taken on the basis of models. The question is not to use or ignore models. The question is only a choice among alternative models. If AI is to meet or surpass human intelligence, then the researchers behind it believe it should be able to make mental models, too. World Labs Li has been working on this through World Labs, which she cofounded in 2024 with an initial backing of $230 million from venture firms like Andreessen Horowitz, New Enterprise Associates, and Radical Ventures. "We aim to lift AI models from the 2D plane of pixels to full 3D worlds - both virtual and real - endowing them with spatial intelligence as rich as our own," World Labs says on its website. Li said on the "No Priors" podcast in June that spatial intelligence is "the ability to understand, reason, interact, and generate 3D worlds," given that the world is fundamentally three-dimensional. Li said she sees applications for world models in creative fields, robotics, or any area that warrants infinite universes. The challenge of building world...

Preview: ~500 words