Google's Genie 4 World Model Adds Audio, More Realistic Worlds, and Real-Time Steering

Google DeepMind released its newest world model, Genie 4, in research preview on Tuesday. The latest model generates the most photorealistic and interactive worlds yet, allowing users to type prompts mid-simulation to change the scene. It also generates real-time audio that matches the simulated world, including music, ambient noise, and interactive dialogue.

Arriving in a Consolidating Market

The release comes a couple of months after AI video startup Runway launched its second world model, GWM Worlds 2, a playable model that responds dynamically to user inputs and includes audio generation. Meanwhile, the world model industry is beginning to consolidate. Fei-Fei Li's World Labs is being acquired by AMD, Amazon is in talks to buy Decart, and Rivian has acquired Third Dimension to bring simulation capabilities in-house for self-driving testing. Genie 4 is Google's latest move to stay ahead in this crowded field.

What's New in Genie 4

Google released its predecessor, Genie 3, as a research preview a little over a year ago. Beyond being far more photorealistic and featuring audio, Genie 4 supports freeform edits—such as changing the weather or camera angle in real time, or adding new people or objects. It can maintain a consistent scene for up to two minutes, double its predecessor's limit, and offers better physics and more human-like characters.

Gemini as Director, Genie as Renderer

Gemini, Google's AI model, is now more closely integrated with Genie 4, acting as the director that plans steps and checks frames while Genie creates the visuals. In a research experiment tied to Gemini Robotics, a Gemini 4 Argon model broke tasks into steps that Genie 4 simulated in real time. The experiment demonstrates a possible way to test robot scenarios in simulation before deploying them in the real world.

"What we'll increasingly see in the future are these hybrid world models, where you have a combination of Gemini writing code and text, and Genie ingesting that in real time and simulating reality at different levels of abstraction," Tejas Kulkarni, research director at Google, said during a briefing.

Commercial Use Cases: Gaming, Learning, Robotics—and Filmmaking

As with other world models, the pitched commercial use cases include gaming, interactive learning, and robotics. One speculative example of interactive learning would be entering a painting of a historical scene and chatting with the painting's subjects about the event portrayed.

But "trusted testers"—the people who get first access to Google's cutting-edge products—are pointing to a possible new use: prototyping films.

"The real-time aspect becomes most valuable in the areas where keeping a state of flow in the creativity space becomes incredibly useful," Diego Rivas, a Google DeepMind engineer, said during a press briefing. "For example, in the ideation phase, in the storyboarding phase—anytime you're in need of rapid ideation."

Google also offers Veo, its AI video generation model that produces high-definition clips with built-in synchronized audio from text, image, or video prompts—a tool that has been growing in popularity among filmmakers. Rivas added that users could, eventually, rely on the two tools together to move from rapid ideation in Genie 4 to polished output in Veo.

via TechCrunch AI