Top Highlights
- AI researchers believe large language models are limited in understanding physical actions, prompting focus on developing “world models” trained on visual and action data.
- Worldmodeldata, a UK startup advised by Yann LeCun, is curating vast video game data to help train these world models, addressing the lack of real-world physics data.
- Video game environments offer diverse, abundant data capturing cause-and-effect, making them ideal for training models on handling real-world scenarios and corner cases.
- The startup has licensed nearly 1 million hours of gaming data and plans to involve players for future data collection, aiming to improve AI’s real-world navigation and manipulation skills.
AI Learns from Gaming Mishaps and Successes
Many people play video games with simple moves, like using a thumbstick or pressing buttons. Surprisingly, these actions can teach AI how to behave in the real world. A British startup is betting that games can be a rich source of training data. The idea is that AI can learn how players solve problems, make decisions, and respond to different situations. This approach offers a new way for AI to understand physics and physical actions. As a result, AI systems could become better at tasks like controlling robots or autonomous vehicles. This method also makes data collection easier, since video games generate huge amounts of behavior data every day.
Training AI to Understand Real-World Physics
Current AI models struggle with moving and manipulating objects accurately. Large language models, which learn from text, cannot easily navigate the physical world. To fix that, scientists focus on building “world models” that learn from visual data and actions. These models need to understand cause and effect — what happens after a specific move. For example, in a factory, an AI must know how tightly to grip an object or how much force to apply. Unlike text, videos of real-world environments are harder to gather. The startup uses video game data, which offers plentiful examples of varied actions and physics. This data helps AI develop a better sense of how the world works.
Breaking Barriers with Video Game Data
Gathering real-world training data is expensive and limited. Trying to record humans and robots in real environments provides only small amounts of data. Additionally, it often misses rare or unexpected situations. Video game data presents a solution. Games simulate diverse scenarios with detailed visual and action information. By collecting this data, AI can learn how to handle tricky situations and unexpected events. This approach can improve safety and performance in real-world applications, such as autonomous cars and robots. The startup has already licensed nearly a million hours of gaming footage. They also plan to involve players directly, giving them a chance to earn rewards and contribute to AI development.
Stay Ahead with the Latest Tech Trends
Stay informed on the revolutionary breakthroughs in Quantum Computing research.
Stay inspired by the vast knowledge available on Wikipedia.
AITechV1
